在kubernetes中使用harbor和NFS

在kubernetes中使用harbor和NFS

evobot 869 2021-12-07

在kubernetes中使用harbor

在master节点上执行以下操作:

  1. 创建secret:

    [root@master1 ~]# kubectl create secret docker-registry evobot-secret --docker-server=harbor.evobot.cn --docker-username=evobot --docker-password=harbor-admin
    secret/evobot-secret created
    
    

    创建完成后,可以使用kubectl get secret查看secret:

    [root@master1 ~]# kubectl get secret
    NAME                  TYPE                                  DATA   AGE
    default-token-hmrwr   kubernetes.io/service-account-token   3      2d5h
    evobot-secret         kubernetes.io/dockerconfigjson        1      16s
    
  2. 定义一个pod

    首先需要在harbor私有仓库里推送一个httpd的镜像,地址为harbor.evobot.cn/evobot/httpd:latest

    docker pull httpd
    docker tag httpd harbor.evobot.cn/evobot/httpd:latest
    docker push harbor.evobot.cn/evobot/httpd:latest
    

    创建pod的httpd.yaml文件

    apiVersion: v1
    kind: Pod
    metadata:
      name: httpd-pod
    spec:
      containers:
      - image: harbor.evobot.cn/evobot/httpd:latest
        name: httpd-pod
      imagePullSecrets:
        - name: evobot-secret
    

    使用kubectl create -f httpd.yaml创建pod,查看pod详情:

    [root@master1 kuberstu]# kubectl get pod
    NAME        READY   STATUS    RESTARTS   AGE
    httpd-pod   1/1     Running   0          104s
    
    [root@master1 kuberstu]# kubectl describe pod httpd
    Name:         httpd-pod
    Namespace:    default
    Priority:     0
    Node:         192.168.139.130/192.168.139.130
    Start Time:   Thu, 02 Dec 2021 20:45:49 +0800
    Labels:       <none>
    Annotations:  <none>
    Status:       Running
    IP:           172.20.3.4
    IPs:
      IP:  172.20.3.4
    Containers:
      httpd-pod:
        Container ID:   docker://f262209c5d640c60040c39aca77c393967cab5d855b05520c5b9e0bf8ed74b92
        Image:          harbor.evobot.cn/evobot/httpd:latest
        Image ID:       docker-pullable://harbor.evobot.cn/evobot/httpd@sha256:24d492e04f02881adcc1d7543b0251754a2be6a24c75aae7a008fdae767b7337
        Port:           <none>
        Host Port:      <none>
        State:          Running
          Started:      Thu, 02 Dec 2021 20:47:09 +0800
        Ready:          True
        Restart Count:  0
        Environment:    <none>
        Mounts:
          /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-9nvpp (ro)
    Conditions:
      Type              Status
      Initialized       True
      Ready             True
      ContainersReady   True
      PodScheduled      True
    Volumes:
      kube-api-access-9nvpp:
        Type:                    Projected (a volume that contains injected data from multiple sources)
        TokenExpirationSeconds:  3607
        ConfigMapName:           kube-root-ca.crt
        ConfigMapOptional:       <nil>
        DownwardAPI:             true
    QoS Class:                   BestEffort
    Node-Selectors:              <none>
    Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                                 node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
    Events:
      Type    Reason     Age   From               Message
      ----    ------     ----  ----               -------
      Normal  Scheduled  110s  default-scheduler  Successfully assigned default/httpd-pod to 192.168.139.130
      Normal  Pulling    105s  kubelet            Pulling image "harbor.evobot.cn/evobot/httpd:latest"
      Normal  Pulled     31s   kubelet            Successfully pulled image "harbor.evobot.cn/evobot/httpd:latest" in 1m14.606130931s
      Normal  Created    31s   kubelet            Created container httpd-pod
      Normal  Started    30s   kubelet            Started container httpd-pod
    
    

kubernetes使用NFS

Persistent Volume(持久化卷)简称PV,是一个K8S资源的对象,我们可以单独创建一个PV,它不和Pod直接产生关系,而是通过Persistent Volume Claim即PVC来实现动态绑定,我们会在Pod定义里指定创建好的PVC,然后PVC会根据Pod的要求去自动绑定合适的PV给Pod使用。

持久化卷下PV和PVC的概念

Persistent Volume(PV)是由管理员设置的存储,它是集群的一部分,就像节点是集群中的资源一样,PV也是集群中的资源。PV是Volume之类的卷插件,但具有独立于使用PV的Pod的生命周期,此API对象包含存储实现的细节,即NFS,iSCSI或特定于云供应商的存储节点。

Persistent Volume Claim(PVC)是用户存储的请求,它与Pod相似,pod消耗节点资源,PVC消耗PV资源,Pod可以请求特定级别的资源(CPU和内存),PVC声明可以请求特定大小和访问模式,例如可以以读/写一次或只读多次模式挂载)。

与普通Volume的区别:

普通Volume和使用它的Pod之间是一种静态绑定关系,在定义Pod的文件里,同时定义了它使用的Volume。Volume是Pod的附属品,我们无法单独创建一个Volume,因为它不是一个独立的K8S资源对象。

使用NFS创建和使用pvc

搭建NFS

  1. 准备一台机器,安装NFS服务:

    yum install nfs-utils
    # vim /etc/exportfs
    /data/shared 192.168.139.0/24(sync,rw,no_root_squash)
    
    systemctl start nfs
    systemctl enable nfs
    
  2. 在node节点上测试nfs

    [root@node1 ~]# showmount -e 192.168.139.33
    Export list for 192.168.139.33:
    /data/shared  192.168.139.0/24
    

创建pv及pvc

  1. 在master节点上创建mypv.yaml

    apiVersion: v1
    kind: PersistentVolume
    metadata:
      name: pv01
    spec:
      capacity:
        storage: 10Gi
      accessModes:
        - ReadWriteMany
      nfs:
        path: /data/shared
        server: 192.168.139.33
    

    创建和查看pv:

    [root@master1 kuberstu]# kubectl create -f mypv.yaml
    persistentvolume/pv01 created
    
    [root@master1 kuberstu]# kubectl get pv
    NAME         CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM   STORAGECLASS    REASON   AGE
    pv01         10Gi       RWX            Retain           Available                                    55s
    
    

    状态为Available,这是因为其还没有被绑定到任何PVC上,当定义完pvc后,就可以自动绑定了。

  2. 在master上创建mypvc.yaml文件:

    kind: PersistentVolumeClaim
    apiVersion: v1
    metadata:
      name: myclaim
    spec:
      accessModes:
        - ReadWriteMany
      resources:
        requests:
          storage: 8Gi
    

    创建和查看pvc:

    [root@master1 kuberstu]# kubectl create -f mypvc.yaml
    persistentvolumeclaim/myclaim created
    
    [root@master1 kuberstu]# kubectl get pvc
    NAME      STATUS   VOLUME   CAPACITY   ACCESS MODES   STORAGECLASS   AGE
    myclaim   Bound    pv01     10Gi       RWX                           5s
    
    

    可以看到pvc状态为Bound,绑定了pv01,对于k8s来说,在pvc的yaml里,不需要指定要使用的pv,k8s会将pvc和pv自动关联。

pod使用pvc

  1. 定义pod,创建pvpod.yaml

    apiVersion: v1
    kind: Pod
    metadata:
      name: httpd-pvpod
    spec:
      containers:
      - image: httpd
        name: httpd-withpvc-pod
        imagePullPolicy: Always
        volumeMounts:
        - mountPath: "/usr/local/apache2/htdocs/"
          name: httpd-volume
      volumes:
        - name: httpd-volume
          persistentVolumeClaim:
            claimName: myclaim
    
  2. 创建和查看pod

    [root@master1 kuberstu]# kubectl create -f pvpod.yaml
    pod/httpd-pvpod created
    
    [root@master1 kuberstu]# kubectl get pod
    NAME                     READY   STATUS              RESTARTS       AGE
    httpd-pod                1/1     Running             1 (4d5h ago)   4d23h
    httpd-pvpod              0/1     ContainerCreating   0              10s
    
    [root@master1 kuberstu]# kubectl get pod
    NAME                     READY   STATUS    RESTARTS       AGE
    httpd-pod                1/1     Running   1 (4d5h ago)   4d23h
    httpd-pvpod              1/1     Running   0              2m
    

pvc验证

  1. 到NFS共享目录下创建文件:

    root@localhost:/# cd /data/shared/
    
    root@localhost:/data/shared# echo 'Test file' > test.html
    
  2. 进入到httpd-pvpod容器内:

    [root@master1 kuberstu]# kubectl exec -it httpd-pvpod bash
    
    root@httpd-pvpod:/usr/local/apache2/htdocs# cat /usr/local/apache2/htdocs/test.html
    Test file
    
  3. 删除http-pvpod:

    [root@master1 kuberstu]# kubectl delete pod httpd-pvpod
    pod "httpd-pvpod" deleted
    
  4. 重建httpd-pvpod:

    [root@master1 kuberstu]# kubectl create -f pvpod.yaml
    pod/httpd-pvpod created
    
    [root@master1 kuberstu]# kubectl get pod
    NAME                     READY   STATUS    RESTARTS       AGE
    httpd-pod                1/1     Running   1 (4d5h ago)   5d
    httpd-pvpod              1/1     Running   0              5s
    
  5. curl访问httpd-pvpod

    [root@master1 kuberstu]# kubectl get pod httpd-pvpod -o wide
    NAME          READY   STATUS    RESTARTS   AGE   IP            NODE              NOMINATED NODE   READINESS GATES
    httpd-pvpod   1/1     Running   0          42s   172.20.2.16   192.168.139.131   <none>           <none>
    
    [root@master1 kuberstu]# curl 172.20.2.16/test.html
    Test file
    
    


# NFS # k8s