在kubernetes中使用harbor和NFS

在kubernetes中使用harbor和NFS

Administrator 897 2021-12-07

在kubernetes中使用harbor

在master节点上执行以下操作：

  1. 创建secret：

    [root@master1 ~]# kubectl create secret docker-registry evobot-secret --docker-server=harbor.evobot.cn --docker-username=evobot --docker-password=harbor-admin
secret/evobot-secret created

    创建完成后，可以使用kubectl get secret查看secret：

    [root@master1 ~]# kubectl get secret
NAME                  TYPE                                  DATA   AGE
default-token-hmrwr   kubernetes.io/service-account-token   3      2d5h
evobot-secret         kubernetes.io/dockerconfigjson        1      16s

  2. 定义一个pod

    首先需要在harbor私有仓库里推送一个httpd的镜像，地址为harbor.evobot.cn/evobot/httpd:latest

    docker pull httpd
docker tag httpd harbor.evobot.cn/evobot/httpd:latest
docker push harbor.evobot.cn/evobot/httpd:latest

    创建pod的httpd.yaml文件

    apiVersion: v1
kind: Pod
metadata:
  name: httpd-pod
spec:
  containers:
  - image: harbor.evobot.cn/evobot/httpd:latest
    name: httpd-pod
  imagePullSecrets:
    - name: evobot-secret

    使用kubectl create -f httpd.yaml创建pod，查看pod详情：

    [root@master1 kuberstu]# kubectl get pod
NAME        READY   STATUS    RESTARTS   AGE
httpd-pod   1/1     Running   0          104s

[root@master1 kuberstu]# kubectl describe pod httpd
Name:         httpd-pod
Namespace:    default
Priority:     0
Node:         192.168.139.130/192.168.139.130
Start Time:   Thu, 02 Dec 2021 20:45:49 +0800
Labels:       <none>
Annotations:  <none>
Status:       Running
IP:           172.20.3.4
IPs:
  IP:  172.20.3.4
Containers:
  httpd-pod:
    Container ID:   docker://f262209c5d640c60040c39aca77c393967cab5d855b05520c5b9e0bf8ed74b92
    Image:          harbor.evobot.cn/evobot/httpd:latest
    Image ID:       docker-pullable://harbor.evobot.cn/evobot/httpd@sha256:24d492e04f02881adcc1d7543b0251754a2be6a24c75aae7a008fdae767b7337
    Port:           <none>
    Host Port:      <none>
    State:          Running
      Started:      Thu, 02 Dec 2021 20:47:09 +0800
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-9nvpp (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             True
  ContainersReady   True
  PodScheduled      True
Volumes:
  kube-api-access-9nvpp:
    Type:                    Projected (a volume that contains injected data from multiple sources)
    TokenExpirationSeconds:  3607
    ConfigMapName:           kube-root-ca.crt
    ConfigMapOptional:       <nil>
    DownwardAPI:             true
QoS Class:                   BestEffort
Node-Selectors:              <none>
Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
                             node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
Events:
  Type    Reason     Age   From               Message
  ----    ------     ----  ----               -------
  Normal  Scheduled  110s  default-scheduler  Successfully assigned default/httpd-pod to 192.168.139.130
  Normal  Pulling    105s  kubelet            Pulling image "harbor.evobot.cn/evobot/httpd:latest"
  Normal  Pulled     31s   kubelet            Successfully pulled image "harbor.evobot.cn/evobot/httpd:latest" in 1m14.606130931s
  Normal  Created    31s   kubelet            Created container httpd-pod
  Normal  Started    30s   kubelet            Started container httpd-pod

kubernetes使用NFS

Persistent Volume(持久化卷)简称PV，是一个K8S资源的对象，我们可以单独创建一个PV，它不和Pod直接产生关系，而是通过Persistent Volume Claim即PVC来实现动态绑定，我们会在Pod定义里指定创建好的PVC，然后PVC会根据Pod的要求去自动绑定合适的PV给Pod使用。

持久化卷下PV和PVC的概念

Persistent Volume（PV）是由管理员设置的存储，它是集群的一部分，就像节点是集群中的资源一样，PV也是集群中的资源。PV是Volume之类的卷插件，但具有独立于使用PV的Pod的生命周期，此API对象包含存储实现的细节，即NFS,iSCSI或特定于云供应商的存储节点。

Persistent Volume Claim（PVC）是用户存储的请求，它与Pod相似，pod消耗节点资源，PVC消耗PV资源，Pod可以请求特定级别的资源（CPU和内存），PVC声明可以请求特定大小和访问模式，例如可以以读/写一次或只读多次模式挂载）。

与普通Volume的区别：

普通Volume和使用它的Pod之间是一种静态绑定关系，在定义Pod的文件里，同时定义了它使用的Volume。Volume是Pod的附属品，我们无法单独创建一个Volume，因为它不是一个独立的K8S资源对象。

使用NFS创建和使用pvc

搭建NFS

  1. 准备一台机器，安装NFS服务：

    yum install nfs-utils
# vim /etc/exportfs
/data/shared 192.168.139.0/24(sync,rw,no_root_squash)

systemctl start nfs
systemctl enable nfs

  2. 在node节点上测试nfs

    [root@node1 ~]# showmount -e 192.168.139.33
Export list for 192.168.139.33:
/data/shared  192.168.139.0/24

创建pv及pvc

  1. 在master节点上创建mypv.yaml

    apiVersion: v1
kind: PersistentVolume
metadata:
  name: pv01
spec:
  capacity:
    storage: 10Gi
  accessModes:
    - ReadWriteMany
  nfs:
    path: /data/shared
    server: 192.168.139.33

    创建和查看pv：

    [root@master1 kuberstu]# kubectl create -f mypv.yaml
persistentvolume/pv01 created

[root@master1 kuberstu]# kubectl get pv
NAME         CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS      CLAIM   STORAGECLASS    REASON   AGE
pv01         10Gi       RWX            Retain           Available                                    55s

    状态为Available，这是因为其还没有被绑定到任何PVC上，当定义完pvc后，就可以自动绑定了。

  2. 在master上创建mypvc.yaml文件：

    kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: myclaim
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 8Gi

    创建和查看pvc：

    [root@master1 kuberstu]# kubectl create -f mypvc.yaml
persistentvolumeclaim/myclaim created

[root@master1 kuberstu]# kubectl get pvc
NAME      STATUS   VOLUME   CAPACITY   ACCESS MODES   STORAGECLASS   AGE
myclaim   Bound    pv01     10Gi       RWX                           5s

    可以看到pvc状态为Bound，绑定了pv01，对于k8s来说，在pvc的yaml里，不需要指定要使用的pv，k8s会将pvc和pv自动关联。

pod使用pvc

  1. 定义pod，创建pvpod.yaml

    apiVersion: v1
kind: Pod
metadata:
  name: httpd-pvpod
spec:
  containers:
  - image: httpd
    name: httpd-withpvc-pod
    imagePullPolicy: Always
    volumeMounts:
    - mountPath: "/usr/local/apache2/htdocs/"
      name: httpd-volume
  volumes:
    - name: httpd-volume
      persistentVolumeClaim:
        claimName: myclaim

  2. 创建和查看pod

    [root@master1 kuberstu]# kubectl create -f pvpod.yaml
pod/httpd-pvpod created

[root@master1 kuberstu]# kubectl get pod
NAME                     READY   STATUS              RESTARTS       AGE
httpd-pod                1/1     Running             1 (4d5h ago)   4d23h
httpd-pvpod              0/1     ContainerCreating   0              10s

[root@master1 kuberstu]# kubectl get pod
NAME                     READY   STATUS    RESTARTS       AGE
httpd-pod                1/1     Running   1 (4d5h ago)   4d23h
httpd-pvpod              1/1     Running   0              2m

pvc验证

  1. 到NFS共享目录下创建文件：

    root@localhost:/# cd /data/shared/

root@localhost:/data/shared# echo 'Test file' > test.html

  2. 进入到httpd-pvpod容器内：

    [root@master1 kuberstu]# kubectl exec -it httpd-pvpod bash

root@httpd-pvpod:/usr/local/apache2/htdocs# cat /usr/local/apache2/htdocs/test.html
Test file

  3. 删除http-pvpod：

    [root@master1 kuberstu]# kubectl delete pod httpd-pvpod
pod "httpd-pvpod" deleted

  4. 重建httpd-pvpod:

    [root@master1 kuberstu]# kubectl create -f pvpod.yaml
pod/httpd-pvpod created

[root@master1 kuberstu]# kubectl get pod
NAME                     READY   STATUS    RESTARTS       AGE
httpd-pod                1/1     Running   1 (4d5h ago)   5d
httpd-pvpod              1/1     Running   0              5s

  5. curl访问httpd-pvpod

    [root@master1 kuberstu]# kubectl get pod httpd-pvpod -o wide
NAME          READY   STATUS    RESTARTS   AGE   IP            NODE              NOMINATED NODE   READINESS GATES
httpd-pvpod   1/1     Running   0          42s   172.20.2.16   192.168.139.131   <none>           <none>

[root@master1 kuberstu]# curl 172.20.2.16/test.html
Test file
k8s NFS