zoukankan      html  css  js  c++  java
  • Kerbernetes的volume基础应用

                Kerbernetes的volume基础应用

                                         作者:尹正杰

    版权声明:原创作品,谢绝转载!否则将追究法律责任。 

    一.存储卷(Volume)概述

    1>.什么是存储卷

      熟悉Docker架构的同学,想必说到存储卷大家应该都很熟悉了,其实在功能上K8S的存储卷和Docker功能类似,Docker是为了持久化Container容器的数据,而K8S则是为了持久化Pods中的数据。

      由于在一个节点上所有的容器是共享宿主机内核的,而驱动是属于内核的。因此K8s node主机内核需要适配目标存储系统,否则该K8s node主机中的Pod将无法使用目标存储系统(如你们公司自己研发的存储系统)。

      虽然K8S已经内置了很多常用的标准类型存储系统驱动,但对于自研的存储系统需要满足Google定义的CSI(Container Storage Interface)规范:     利用CSI用户可以方便开发自己的Pods存储驱动插件,可以自定义使用任何的类型的存储系统插件。
    [root@node201.yinzhengjie.org.cn ~]# kubectl explain pods.spec.volumes
    KIND:     Pod
    VERSION:  v1
    
    RESOURCE: volumes <[]Object>
    
    DESCRIPTION:
         List of volumes that can be mounted by containers belonging to the pod.
         More info: https://kubernetes.io/docs/concepts/storage/volumes
    
         Volume represents a named volume in a pod that may be accessed by any
         container in the pod.
    
    FIELDS:
       awsElasticBlockStore    <Object>
         AWSElasticBlockStore represents an AWS Disk resource that is attached to a
         kubelet's host machine and then exposed to the pod. More info:
         https://kubernetes.io/docs/concepts/storage/volumes#awselasticblockstore
    
       azureDisk    <Object>
         AzureDisk represents an Azure Data Disk mount on the host and bind mount to
         the pod.
    
       azureFile    <Object>
         AzureFile represents an Azure File Service mount on the host and bind mount
         to the pod.
    
       cephfs    <Object>
         CephFS represents a Ceph FS mount on the host that shares a pod's lifetime
    
       cinder    <Object>
         Cinder represents a cinder volume attached and mounted on kubelets host
         machine. More info: https://examples.k8s.io/mysql-cinder-pd/README.md
    
       configMap    <Object>
         ConfigMap represents a configMap that should populate this volume
    
       csi    <Object>
         CSI (Container Storage Interface) represents storage that is handled by an
         external CSI driver (Alpha feature).
    
       downwardAPI    <Object>
         DownwardAPI represents downward API about the pod that should populate this
         volume
    
       emptyDir    <Object>
         EmptyDir represents a temporary directory that shares a pod's lifetime.
         More info: https://kubernetes.io/docs/concepts/storage/volumes#emptydir
    
       fc    <Object>
         FC represents a Fibre Channel resource that is attached to a kubelet's host
         machine and then exposed to the pod.
    
       flexVolume    <Object>
         FlexVolume represents a generic volume resource that is
         provisioned/attached using an exec based plugin.
    
       flocker    <Object>
         Flocker represents a Flocker volume attached to a kubelet's host machine.
         This depends on the Flocker control service being running
    
       gcePersistentDisk    <Object>
         GCEPersistentDisk represents a GCE Disk resource that is attached to a
         kubelet's host machine and then exposed to the pod. More info:
         https://kubernetes.io/docs/concepts/storage/volumes#gcepersistentdisk
    
       gitRepo    <Object>
         GitRepo represents a git repository at a particular revision. DEPRECATED:
         GitRepo is deprecated. To provision a container with a git repo, mount an
         EmptyDir into an InitContainer that clones the repo using git, then mount
         the EmptyDir into the Pod's container.
    
       glusterfs    <Object>
         Glusterfs represents a Glusterfs mount on the host that shares a pod's
         lifetime. More info: https://examples.k8s.io/volumes/glusterfs/README.md
    
       hostPath    <Object>
         HostPath represents a pre-existing file or directory on the host machine
         that is directly exposed to the container. This is generally used for
         system agents or other privileged things that are allowed to see the host
         machine. Most containers will NOT need this. More info:
         https://kubernetes.io/docs/concepts/storage/volumes#hostpath
    
       iscsi    <Object>
         ISCSI represents an ISCSI Disk resource that is attached to a kubelet's
         host machine and then exposed to the pod. More info:
         https://examples.k8s.io/volumes/iscsi/README.md
    
       name    <string> -required-
         Volume's name. Must be a DNS_LABEL and unique within the pod. More info:
         https://kubernetes.io/docs/concepts/overview/working-with-objects/names/#names
    
       nfs    <Object>
         NFS represents an NFS mount on the host that shares a pod's lifetime More
         info: https://kubernetes.io/docs/concepts/storage/volumes#nfs
    
       persistentVolumeClaim    <Object>
         PersistentVolumeClaimVolumeSource represents a reference to a
         PersistentVolumeClaim in the same namespace. More info:
         https://kubernetes.io/docs/concepts/storage/persistent-volumes#persistentvolumeclaims
    
       photonPersistentDisk    <Object>
         PhotonPersistentDisk represents a PhotonController persistent disk attached
         and mounted on kubelets host machine
    
       portworxVolume    <Object>
         PortworxVolume represents a portworx volume attached and mounted on
         kubelets host machine
    
       projected    <Object>
         Items for all in one resources secrets, configmaps, and downward API
    
       quobyte    <Object>
         Quobyte represents a Quobyte mount on the host that shares a pod's lifetime
    
       rbd    <Object>
         RBD represents a Rados Block Device mount on the host that shares a pod's
         lifetime. More info: https://examples.k8s.io/volumes/rbd/README.md
    
       scaleIO    <Object>
         ScaleIO represents a ScaleIO persistent volume attached and mounted on
         Kubernetes nodes.
    
       secret    <Object>
         Secret represents a secret that should populate this volume. More info:
         https://kubernetes.io/docs/concepts/storage/volumes#secret
    
       storageos    <Object>
         StorageOS represents a StorageOS volume attached and mounted on Kubernetes
         nodes.
    
       vsphereVolume    <Object>
         VsphereVolume represents a vSphere volume attached and mounted on kubelets
         host machine
    
    [root@node201.yinzhengjie.org.cn ~]# 
    [root@node201.yinzhengjie.org.cn ~]# 
    [root@node201.yinzhengjie.org.cn ~]# kubectl explain pods.spec.volumes            #查看Kubernetes内置的存储卷(Volume)类型

    2>.Kubernetes内置支持的存储卷类型

      如下所示,Kubernetes内置支持的存储插件分类如下所示,具体配置方法可以参考官方文档:https://kubernetes.io/docs/concepts/storage/volumes/  
    
      云存储(对于本地维护要求大,交钱就好啦,我们知道对于有的公司来说能用钱解决的问题就不是问题):
        awsElasticBlockStore
        azureDisk
        azureFile
        gcePersistentDisk
        vsphereVolume
        cinder
    
      分布式存储(需要存储工程师来维护):
        cephfs(RedHat)
        glusterfs(RedHat)
        rbd
    
      网络存储:
        nfs(NAS)
        iscsi(SAN)
        fs(SAN)
    
      临时存储:
        gitRepo(deprecated,已弃用)
        emptyDir
    
      本地存储:
        hostPath
        local
    
      特殊存储:
        configMap
        secret
        downwardAPI
    
      持久卷存储:
        persistentVolumeClaim(简称PVC)
    
      自定义存储:
        csi

    3>.使用存储卷(Volumes)的注意事项

      在Kerbernetes集群的Pod上想要使用存储卷(Volumes),需要注意以下事项:
        (1)确保外部存储系统(Storage System)可用;
        (2)需要在Pod级别指明存储系统的存储服务作为存储卷在当前Pod中使用(此时存储卷只是将"k8s.gcr.io/pause"这个Pod作为存储系统的客户端);
        (3)之后,在该Pod中的容器上挂载该存储卷(如果此步骤不做,尽管你做了第二步容器也无法使用存储卷,因为在该容器中没有挂载点)

    二.本地存储卷(hostPath)使用案例(挂在node节点本地的目录)

    1>.编写支持hostPath的存储卷使用案例(参考文档:https://kubernetes.io/docs/concepts/storage/volumes/#hostpath)

    [root@master200.yinzhengjie.org.cn ~]# kubectl explain pods.spec.volumes.hostPath
    KIND:     Pod
    VERSION:  v1
    
    RESOURCE: hostPath <Object>
    
    DESCRIPTION:
         HostPath represents a pre-existing file or directory on the host machine
         that is directly exposed to the container. This is generally used for
         system agents or other privileged things that are allowed to see the host
         machine. Most containers will NOT need this. More info:
         https://kubernetes.io/docs/concepts/storage/volumes#hostpath
    
         Represents a host path mapped into a pod. Host path volumes do not support
         ownership management or SELinux relabeling.
    
    FIELDS:
       path    <string> -required-
         Path of the directory on the host. If the path is a symlink, it will follow
         the link to the real path. More info:
         https://kubernetes.io/docs/concepts/storage/volumes#hostpath
    
       type    <string>
         Type for HostPath Volume Defaults to "" More info:
         https://kubernetes.io/docs/concepts/storage/volumes#hostpath
    
    [root@master200.yinzhengjie.org.cn ~]# 
    [root@master200.yinzhengjie.org.cn ~]# kubectl explain pods.spec.volumes.hostPath
    [root@master200.yinzhengjie.org.cn ~]# vim /yinzhengjie/data/k8s/manifests/basic/volumes/pod-volume-demo.yaml
    [root@master200.yinzhengjie.org.cn ~]# 
    [root@master200.yinzhengjie.org.cn ~]# cat /yinzhengjie/data/k8s/manifests/basic/volumes/pod-volume-demo.yaml
    apiVersion: v1
    kind: Namespace
    metadata:
      name: yinzhengjie-volume
    ---
    apiVersion: v1
    kind: Pod
    metadata:
      name: myapp
      namespace: yinzhengjie-volume
      labels:
        app: myapp
    spec:
      containers:
      - name: myapp
        image: ikubernetes/myapp:v1
        volumeMounts:
        - name: web-storage
          mountPath: /usr/share/nginx/html
          readOnly: true
      volumes:
      - name: web-storage
        hostPath: 
          path: /yinzhengjie/data/volumes/myapp
          type: DirectoryOrCreate
    [root@master200.yinzhengjie.org.cn ~]# 
    [root@master200.yinzhengjie.org.cn ~]# vim /yinzhengjie/data/k8s/manifests/basic/volumes/pod-volume-demo.yaml          #编写pod的yaml格式文件
    [root@master200.yinzhengjie.org.cn ~]# kubectl apply -f /yinzhengjie/data/k8s/manifests/basic/volumes/pod-volume-demo.yaml
    namespace/yinzhengjie-volume created
    pod/myapp created
    [root@master200.yinzhengjie.org.cn ~]# 
    [root@master200.yinzhengjie.org.cn ~]# kubectl get pod -n yinzhengjie-volume
    NAME    READY   STATUS    RESTARTS   AGE
    myapp   1/1     Running   0          33s
    [root@master200.yinzhengjie.org.cn ~]# 
    [root@master200.yinzhengjie.org.cn ~]# 
    [root@master200.yinzhengjie.org.cn ~]# kubectl apply -f /yinzhengjie/data/k8s/manifests/basic/volumes/pod-volume-demo.yaml
    [root@master200.yinzhengjie.org.cn ~]# kubectl describe pods myapp -n yinzhengjie-volume
    Name:         myapp
    Namespace:    yinzhengjie-volume
    Priority:     0
    Node:         node203.yinzhengjie.org.cn/172.200.1.203
    Start Time:   Sun, 09 Feb 2020 06:51:59 +0800
    Labels:       app=myapp
    Annotations:  kubectl.kubernetes.io/last-applied-configuration:
                    {"apiVersion":"v1","kind":"Pod","metadata":{"annotations":{},"labels":{"app":"myapp"},"name":"myapp","namespace":"yinzhengjie-volume"},"sp...
    Status:       Running
    IP:           10.244.3.9
    IPs:
      IP:  10.244.3.9
    Containers:
      myapp:
        Container ID:   docker://79046dcdb5afc659ffcc910305dc98ccb8988501e0d52aea96d133443e2bd552
        Image:          ikubernetes/myapp:v1
        Image ID:       docker-pullable://ikubernetes/myapp@sha256:9c3dc30b5219788b2b8a4b065f548b922a34479577befb54b03330999d30d513
        Port:           <none>
        Host Port:      <none>
        State:          Running
          Started:      Sun, 09 Feb 2020 06:52:00 +0800
        Ready:          True
        Restart Count:  0
        Environment:    <none>
        Mounts:
          /usr/share/nginx/html from web-storage (ro)
          /var/run/secrets/kubernetes.io/serviceaccount from default-token-xkkkg (ro)
    Conditions:
      Type              Status
      Initialized       True 
      Ready             True 
      ContainersReady   True 
      PodScheduled      True 
    Volumes:
      web-storage:
        Type:          HostPath (bare host directory volume)
        Path:          /yinzhengjie/data/volumes/myapp
        HostPathType:  DirectoryOrCreate
      default-token-xkkkg:
        Type:        Secret (a volume populated by a Secret)
        SecretName:  default-token-xkkkg
        Optional:    false
    QoS Class:       BestEffort
    Node-Selectors:  <none>
    Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                     node.kubernetes.io/unreachable:NoExecute for 300s
    Events:
      Type    Reason     Age        From                                 Message
      ----    ------     ----       ----                                 -------
      Normal  Scheduled  <unknown>  default-scheduler                    Successfully assigned yinzhengjie-volume/myapp to node203.yinzhengjie.org.cn
      Normal  Pulled     3m9s       kubelet, node203.yinzhengjie.org.cn  Container image "ikubernetes/myapp:v1" already present on machine
      Normal  Created    3m9s       kubelet, node203.yinzhengjie.org.cn  Created container myapp
      Normal  Started    3m9s       kubelet, node203.yinzhengjie.org.cn  Started container myapp
    [root@master200.yinzhengjie.org.cn ~]# 
    [root@master200.yinzhengjie.org.cn ~]# 
    [root@master200.yinzhengjie.org.cn ~]# kubectl describe pods myapp -n yinzhengjie-volume

    2>.验证Pod服务是否正常访问

    [root@master200.yinzhengjie.org.cn ~]# kubectl get pod -n yinzhengjie-volume -o wide
    NAME    READY   STATUS    RESTARTS   AGE     IP           NODE                         NOMINATED NODE   READINESS GATES
    myapp   1/1     Running   0          4m17s   10.244.3.9   node203.yinzhengjie.org.cn   <none>           <none>
    [root@master200.yinzhengjie.org.cn ~]# 
    [root@master200.yinzhengjie.org.cn ~]# ssh node203.yinzhengjie.org.cn
    root@node203.yinzhengjie.org.cn's password: 
    Last login: Sat Feb  8 11:10:40 2020 from 172.200.0.1
    [root@node203.yinzhengjie.org.cn ~]# 
    [root@node203.yinzhengjie.org.cn ~]# ls /
    bin  boot  dev  etc  home  lib  lib64  media  mnt  opt  proc  root  run  sbin  srv  sys  tmp  usr  var  yinzhengjie
    [root@node203.yinzhengjie.org.cn ~]# 
    [root@node203.yinzhengjie.org.cn ~]# ls /yinzhengjie/data/volumes/myapp/
    [root@node203.yinzhengjie.org.cn ~]# 
    [root@node203.yinzhengjie.org.cn ~]# ll /yinzhengjie/data/volumes/myapp/
    total 0
    [root@node203.yinzhengjie.org.cn ~]# 
    [root@node203.yinzhengjie.org.cn ~]# curl 10.244.3.9
    <html>
    <head><title>403 Forbidden</title></head>
    <body bgcolor="white">
    <center><h1>403 Forbidden</h1></center>
    <hr><center>nginx/1.12.2</center>
    </body>
    </html>
    [root@node203.yinzhengjie.org.cn ~]# 
    [root@node203.yinzhengjie.org.cn ~]# echo "<h1>My volumes Page</h1>" > /yinzhengjie/data/volumes/myapp/index.html
    [root@node203.yinzhengjie.org.cn ~]# 
    [root@node203.yinzhengjie.org.cn ~]# ll /yinzhengjie/data/volumes/myapp/
    total 4
    -rw-r--r-- 1 root root 25 Feb  9 06:59 index.html
    [root@node203.yinzhengjie.org.cn ~]# 
    [root@node203.yinzhengjie.org.cn ~]# cat /yinzhengjie/data/volumes/myapp/index.html 
    <h1>My volumes Page</h1>
    [root@node203.yinzhengjie.org.cn ~]# 
    [root@node203.yinzhengjie.org.cn ~]# curl 10.244.3.9
    <h1>My volumes Page</h1>
    [root@node203.yinzhengjie.org.cn ~]# 
    [root@node203.yinzhengjie.org.cn ~]# 
    [root@node203.yinzhengjie.org.cn ~]# 
    [root@master200.yinzhengjie.org.cn ~]# kubectl get pod -n yinzhengjie-volume -o wide

    3>.优化hostPath方案

      hostPath存在的风险剖析:
        当我们手动删除Pod时,如果新建的Pod不在之前的那台k8s node主机上,则数据全部丢失,因此我们可以绑定Pod在某一台k8s node主机上,这样新建的Pod数据还是可以在该主机上获取到。
        综上所述,我们可以总结为hostPath只能维持节点的数据存储的持久性,当节点宕机时数据依旧是访问不了啦,而且Pod只能被调度到拥有相同数据的node主机。

      优化hostPath方案1:
        我们将web数据在所有K8S node主机上使用rsync同步一份数据到其它节点,这样无论Pod被调度到K8s集群的任何一台服务器数据都不会丢失,但是该方案有个很明显的缺点就是浪费存储空间。

      优化hostPath方案2:
        我们可以使用分布式存储系统,然后在K8s node主机上配置上挂载,这样被写入的数据虽说是本地的存储目录,但实际上缺存储到分布式文件系统上了,相比上一种解决方案更加节省空间,而且还可以借助存储系统本身的可用性保证数据不丢失。

      优化hostPath方案3:
        就是直接在编辑Pod的yaml文件时指定每次启动该容器时只能(绑定)运行在某一台节点上,这样每次启动数据时数据均不会丢失,该方案很明显的缺陷就是当该节点宕机时,则Pod长时间无法完成调度。

    4>.另一种本地存储卷(local)方案(参考文档:https://kubernetes.io/docs/concepts/storage/volumes/#local)

    三.临时存储卷(emptyDir)使用案例(一般用作缓存空间)

    1>. 编写支持hostPath的存储卷使用案例(参考文档:https://kubernetes.io/docs/concepts/storage/volumes/#emptydir)

    [root@master200.yinzhengjie.org.cn ~]# kubectl explain pods.spec.volumes.emptyDir
    KIND:     Pod
    VERSION:  v1
    
    RESOURCE: emptyDir <Object>
    
    DESCRIPTION:
         EmptyDir represents a temporary directory that shares a pod's lifetime.
         More info: https://kubernetes.io/docs/concepts/storage/volumes#emptydir
    
         Represents an empty directory for a pod. Empty directory volumes support
         ownership management and SELinux relabeling.
    
    FIELDS:
       medium    <string>
         What type of storage medium should back this directory. The default is ""
         which means to use the node's default medium. Must be an empty string
         (default) or Memory. More info:
         https://kubernetes.io/docs/concepts/storage/volumes#emptydir
    
       sizeLimit    <string>
         Total amount of local storage required for this EmptyDir volume. The size
         limit is also applicable for memory medium. The maximum usage on memory
         medium EmptyDir would be the minimum value between the SizeLimit specified
         here and the sum of memory limits of all containers in a pod. The default
         is nil which means that the limit is undefined. More info:
         http://kubernetes.io/docs/user-guide/volumes#emptydir
    
    [root@master200.yinzhengjie.org.cn ~]# 
    [root@master200.yinzhengjie.org.cn ~]# 
    [root@master200.yinzhengjie.org.cn ~]# kubectl explain pods.spec.volumes.emptyDir
    [root@master200.yinzhengjie.org.cn ~]# vim /yinzhengjie/data/k8s/manifests/basic/volumes/pod-volume-demo2.yaml 
    [root@master200.yinzhengjie.org.cn ~]# 
    [root@master200.yinzhengjie.org.cn ~]# cat /yinzhengjie/data/k8s/manifests/basic/volumes/pod-volume-demo2.yaml 
    apiVersion: v1
    kind: Pod
    metadata:
      name: vol-emptydir-pod
      namespace: yinzhengjie-volume
    spec:
      volumes:
      - name: html
        emptyDir: {}
      containers:
      - name: myapp
        image: ikubernetes/myapp:v1
        volumeMounts:
        - name: html
          mountPath: /usr/share/nginx/html
      - name: pagegen
        image: alpine
        volumeMounts:
        - name: html
          mountPath: /html
        command: ["/bin/sh", "-c"]
        args:
        - while true; do
            echo $(hostname) $(date) >> /html/index.html;
            sleep 10;
          done
    [root@master200.yinzhengjie.org.cn ~]# 
    [root@master200.yinzhengjie.org.cn ~]# vim /yinzhengjie/data/k8s/manifests/basic/volumes/pod-volume-demo2.yaml
    [root@master200.yinzhengjie.org.cn ~]# kubectl apply -f /yinzhengjie/data/k8s/manifests/basic/volumes/pod-volume-demo2.yaml 
    pod/vol-emptydir-pod created
    [root@master200.yinzhengjie.org.cn ~]# 
    [root@master200.yinzhengjie.org.cn ~]# kubectl get pods -n yinzhengjie-volume
    NAME               READY   STATUS    RESTARTS   AGE
    myapp              1/1     Running   0          106m
    vol-emptydir-pod   2/2     Running   0          17s
    [root@master200.yinzhengjie.org.cn ~]# 
    [root@master200.yinzhengjie.org.cn ~]# 
    [root@master200.yinzhengjie.org.cn ~]# kubectl apply -f /yinzhengjie/data/k8s/manifests/basic/volumes/pod-volume-demo2.yaml
    [root@master200.yinzhengjie.org.cn ~]# kubectl describe pods vol-emptydir-pod -n yinzhengjie-volume
    Name:         vol-emptydir-pod
    Namespace:    yinzhengjie-volume
    Priority:     0
    Node:         node201.yinzhengjie.org.cn/172.200.1.201
    Start Time:   Sun, 09 Feb 2020 09:11:22 +0800
    Labels:       <none>
    Annotations:  kubectl.kubernetes.io/last-applied-configuration:
                    {"apiVersion":"v1","kind":"Pod","metadata":{"annotations":{},"name":"vol-emptydir-pod","namespace":"yinzhengjie-volume"},"spec":{"containe...
    Status:       Running
    IP:           10.244.1.22
    IPs:
      IP:  10.244.1.22
    Containers:
      myapp:
        Container ID:   docker://f885946b5e50a1b02a47b25fb079113001ec12eab334bd10e13b0d430f2846b5
        Image:          ikubernetes/myapp:v1
        Image ID:       docker-pullable://ikubernetes/myapp@sha256:9c3dc30b5219788b2b8a4b065f548b922a34479577befb54b03330999d30d513
        Port:           <none>
        Host Port:      <none>
        State:          Running
          Started:      Sun, 09 Feb 2020 09:11:22 +0800
        Ready:          True
        Restart Count:  0
        Environment:    <none>
        Mounts:
          /usr/share/nginx/html from html (rw)
          /var/run/secrets/kubernetes.io/serviceaccount from default-token-bxflv (ro)
      pagegen:
        Container ID:  docker://cd899fb80117212e61219dbc29df58ca4d67b6c2e3b84abe5bcf4499827d69f7
        Image:         alpine
        Image ID:      docker-pullable://alpine@sha256:ab00606a42621fb68f2ed6ad3c88be54397f981a7b70a79db3d1172b11c4367d
        Port:          <none>
        Host Port:     <none>
        Command:
          /bin/sh
          -c
        Args:
          while true; do echo $(hostname) $(date) >> /html/index.html; sleep 10; done
        State:          Running
          Started:      Sun, 09 Feb 2020 09:11:25 +0800
        Ready:          True
        Restart Count:  0
        Environment:    <none>
        Mounts:
          /html from html (rw)
          /var/run/secrets/kubernetes.io/serviceaccount from default-token-bxflv (ro)
    Conditions:
      Type              Status
      Initialized       True 
      Ready             True 
      ContainersReady   True 
      PodScheduled      True 
    Volumes:
      html:
        Type:       EmptyDir (a temporary directory that shares a pod's lifetime)
        Medium:     
        SizeLimit:  <unset>
      default-token-bxflv:
        Type:        Secret (a volume populated by a Secret)
        SecretName:  default-token-bxflv
        Optional:    false
    QoS Class:       BestEffort
    Node-Selectors:  <none>
    Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                     node.kubernetes.io/unreachable:NoExecute for 300s
    Events:
      Type    Reason     Age        From                                 Message
      ----    ------     ----       ----                                 -------
      Normal  Scheduled  <unknown>  default-scheduler                    Successfully assigned yinzhengjie-volume/vol-emptydir-pod to node201.yinzhengjie.org.cn
      Normal  Pulled     118s       kubelet, node201.yinzhengjie.org.cn  Container image "ikubernetes/myapp:v1" already present on machine
      Normal  Created    118s       kubelet, node201.yinzhengjie.org.cn  Created container myapp
      Normal  Started    118s       kubelet, node201.yinzhengjie.org.cn  Started container myapp
      Normal  Pulling    118s       kubelet, node201.yinzhengjie.org.cn  Pulling image "alpine"
      Normal  Pulled     115s       kubelet, node201.yinzhengjie.org.cn  Successfully pulled image "alpine"
      Normal  Created    115s       kubelet, node201.yinzhengjie.org.cn  Created container pagegen
      Normal  Started    115s       kubelet, node201.yinzhengjie.org.cn  Started container pagegen
    [root@master200.yinzhengjie.org.cn ~]# 
    [root@master200.yinzhengjie.org.cn ~]# kubectl describe pods vol-emptydir-pod -n yinzhengjie-volume

    2>.验证Pod服务是否正常访问

    四.网络(nfs)存储案例

    1>.构建NFS网络文件系统(NFS默认是不具有高可用性的,他需要借助于rsync类似的服务实现数据备份,生产环境建议大家使用本身就具有高可用性的文件系统,我这里为了试验方便就采用nfs)

    [root@test110.yinzhengjie.org.cn ~]# yum -y install nfs-utils
    Loaded plugins: fastestmirror
    Loading mirror speeds from cached hostfile
     * base: mirror.bit.edu.cn
     * extras: mirrors.tuna.tsinghua.edu.cn
     * updates: mirrors.tuna.tsinghua.edu.cn
    base                                                                                                                                                                                                                                                   | 3.6 kB  00:00:00     
    docker-ce-stable                                                                                                                                                                                                                                       | 3.5 kB  00:00:00     
    extras                                                                                                                                                                                                                                                 | 2.9 kB  00:00:00     
    updates                                                                                                                                                                                                                                                | 2.9 kB  00:00:00     
    updates/7/x86_64/primary_db                                                                                                                                                                                                                            | 6.7 MB  00:00:01     
    Resolving Dependencies
    --> Running transaction check
    ---> Package nfs-utils.x86_64 1:1.3.0-0.65.el7 will be installed
    --> Processing Dependency: libtirpc >= 0.2.4-0.7 for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
    --> Processing Dependency: gssproxy >= 0.7.0-3 for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
    --> Processing Dependency: rpcbind for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
    --> Processing Dependency: quota for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
    --> Processing Dependency: libnfsidmap for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
    --> Processing Dependency: keyutils for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
    --> Processing Dependency: libtirpc.so.1()(64bit) for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
    --> Processing Dependency: libnfsidmap.so.0()(64bit) for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
    --> Running transaction check
    ---> Package gssproxy.x86_64 0:0.7.0-26.el7 will be installed
    --> Processing Dependency: libini_config >= 1.3.1-31 for package: gssproxy-0.7.0-26.el7.x86_64
    --> Processing Dependency: libverto-module-base for package: gssproxy-0.7.0-26.el7.x86_64
    --> Processing Dependency: libref_array.so.1(REF_ARRAY_0.1.1)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
    --> Processing Dependency: libini_config.so.3(INI_CONFIG_1.2.0)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
    --> Processing Dependency: libini_config.so.3(INI_CONFIG_1.1.0)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
    --> Processing Dependency: libref_array.so.1()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
    --> Processing Dependency: libini_config.so.3()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
    --> Processing Dependency: libcollection.so.2()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
    --> Processing Dependency: libbasicobjects.so.0()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
    ---> Package keyutils.x86_64 0:1.5.8-3.el7 will be installed
    ---> Package libnfsidmap.x86_64 0:0.25-19.el7 will be installed
    ---> Package libtirpc.x86_64 0:0.2.4-0.16.el7 will be installed
    ---> Package quota.x86_64 1:4.01-19.el7 will be installed
    --> Processing Dependency: quota-nls = 1:4.01-19.el7 for package: 1:quota-4.01-19.el7.x86_64
    --> Processing Dependency: tcp_wrappers for package: 1:quota-4.01-19.el7.x86_64
    ---> Package rpcbind.x86_64 0:0.2.0-48.el7 will be installed
    --> Running transaction check
    ---> Package libbasicobjects.x86_64 0:0.1.1-32.el7 will be installed
    ---> Package libcollection.x86_64 0:0.7.0-32.el7 will be installed
    ---> Package libini_config.x86_64 0:1.3.1-32.el7 will be installed
    --> Processing Dependency: libpath_utils.so.1(PATH_UTILS_0.2.1)(64bit) for package: libini_config-1.3.1-32.el7.x86_64
    --> Processing Dependency: libpath_utils.so.1()(64bit) for package: libini_config-1.3.1-32.el7.x86_64
    ---> Package libref_array.x86_64 0:0.1.5-32.el7 will be installed
    ---> Package libverto-libevent.x86_64 0:0.2.5-4.el7 will be installed
    ---> Package quota-nls.noarch 1:4.01-19.el7 will be installed
    ---> Package tcp_wrappers.x86_64 0:7.6-77.el7 will be installed
    --> Running transaction check
    ---> Package libpath_utils.x86_64 0:0.2.1-32.el7 will be installed
    --> Finished Dependency Resolution
    
    Dependencies Resolved
    
    ==============================================================================================================================================================================================================================================================================
     Package                                                                 Arch                                                         Version                                                                Repository                                                  Size
    ==============================================================================================================================================================================================================================================================================
    Installing:
     nfs-utils                                                               x86_64                                                       1:1.3.0-0.65.el7                                                       base                                                       412 k
    Installing for dependencies:
     gssproxy                                                                x86_64                                                       0.7.0-26.el7                                                           base                                                       110 k
     keyutils                                                                x86_64                                                       1.5.8-3.el7                                                            base                                                        54 k
     libbasicobjects                                                         x86_64                                                       0.1.1-32.el7                                                           base                                                        26 k
     libcollection                                                           x86_64                                                       0.7.0-32.el7                                                           base                                                        42 k
     libini_config                                                           x86_64                                                       1.3.1-32.el7                                                           base                                                        64 k
     libnfsidmap                                                             x86_64                                                       0.25-19.el7                                                            base                                                        50 k
     libpath_utils                                                           x86_64                                                       0.2.1-32.el7                                                           base                                                        28 k
     libref_array                                                            x86_64                                                       0.1.5-32.el7                                                           base                                                        27 k
     libtirpc                                                                x86_64                                                       0.2.4-0.16.el7                                                         base                                                        89 k
     libverto-libevent                                                       x86_64                                                       0.2.5-4.el7                                                            base                                                       8.9 k
     quota                                                                   x86_64                                                       1:4.01-19.el7                                                          base                                                       179 k
     quota-nls                                                               noarch                                                       1:4.01-19.el7                                                          base                                                        90 k
     rpcbind                                                                 x86_64                                                       0.2.0-48.el7                                                           base                                                        60 k
     tcp_wrappers                                                            x86_64                                                       7.6-77.el7                                                             base                                                        78 k
    
    Transaction Summary
    ==============================================================================================================================================================================================================================================================================
    Install  1 Package (+14 Dependent packages)
    
    Total download size: 1.3 M
    Installed size: 3.6 M
    Downloading packages:
    (1/15): keyutils-1.5.8-3.el7.x86_64.rpm                                                                                                                                                                                                                |  54 kB  00:00:00     
    (2/15): libcollection-0.7.0-32.el7.x86_64.rpm                                                                                                                                                                                                          |  42 kB  00:00:00     
    (3/15): libini_config-1.3.1-32.el7.x86_64.rpm                                                                                                                                                                                                          |  64 kB  00:00:00     
    (4/15): gssproxy-0.7.0-26.el7.x86_64.rpm                                                                                                                                                                                                               | 110 kB  00:00:00     
    (5/15): libnfsidmap-0.25-19.el7.x86_64.rpm                                                                                                                                                                                                             |  50 kB  00:00:00     
    (6/15): libref_array-0.1.5-32.el7.x86_64.rpm                                                                                                                                                                                                           |  27 kB  00:00:00     
    (7/15): libpath_utils-0.2.1-32.el7.x86_64.rpm                                                                                                                                                                                                          |  28 kB  00:00:00     
    (8/15): libverto-libevent-0.2.5-4.el7.x86_64.rpm                                                                                                                                                                                                       | 8.9 kB  00:00:00     
    (9/15): libtirpc-0.2.4-0.16.el7.x86_64.rpm                                                                                                                                                                                                             |  89 kB  00:00:00     
    (10/15): libbasicobjects-0.1.1-32.el7.x86_64.rpm                                                                                                                                                                                                       |  26 kB  00:00:00     
    (11/15): quota-4.01-19.el7.x86_64.rpm                                                                                                                                                                                                                  | 179 kB  00:00:00     
    (12/15): quota-nls-4.01-19.el7.noarch.rpm                                                                                                                                                                                                              |  90 kB  00:00:00     
    (13/15): tcp_wrappers-7.6-77.el7.x86_64.rpm                                                                                                                                                                                                            |  78 kB  00:00:00     
    (14/15): nfs-utils-1.3.0-0.65.el7.x86_64.rpm                                                                                                                                                                                                           | 412 kB  00:00:00     
    (15/15): rpcbind-0.2.0-48.el7.x86_64.rpm                                                                                                                                                                                                               |  60 kB  00:00:00     
    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    Total                                                                                                                                                                                                                                         2.9 MB/s | 1.3 MB  00:00:00     
    Running transaction check
    Running transaction test
    Transaction test succeeded
    Running transaction
      Installing : libbasicobjects-0.1.1-32.el7.x86_64                                                                                                                                                                                                                       1/15 
      Installing : libref_array-0.1.5-32.el7.x86_64                                                                                                                                                                                                                          2/15 
      Installing : libcollection-0.7.0-32.el7.x86_64                                                                                                                                                                                                                         3/15 
      Installing : libtirpc-0.2.4-0.16.el7.x86_64                                                                                                                                                                                                                            4/15 
      Installing : rpcbind-0.2.0-48.el7.x86_64                                                                                                                                                                                                                               5/15 
      Installing : 1:quota-nls-4.01-19.el7.noarch                                                                                                                                                                                                                            6/15 
      Installing : tcp_wrappers-7.6-77.el7.x86_64                                                                                                                                                                                                                            7/15 
      Installing : 1:quota-4.01-19.el7.x86_64                                                                                                                                                                                                                                8/15 
      Installing : keyutils-1.5.8-3.el7.x86_64                                                                                                                                                                                                                               9/15 
      Installing : libnfsidmap-0.25-19.el7.x86_64                                                                                                                                                                                                                           10/15 
      Installing : libpath_utils-0.2.1-32.el7.x86_64                                                                                                                                                                                                                        11/15 
      Installing : libini_config-1.3.1-32.el7.x86_64                                                                                                                                                                                                                        12/15 
      Installing : libverto-libevent-0.2.5-4.el7.x86_64                                                                                                                                                                                                                     13/15 
      Installing : gssproxy-0.7.0-26.el7.x86_64                                                                                                                                                                                                                             14/15 
      Installing : 1:nfs-utils-1.3.0-0.65.el7.x86_64                                                                                                                                                                                                                        15/15 
      Verifying  : libtirpc-0.2.4-0.16.el7.x86_64                                                                                                                                                                                                                            1/15 
      Verifying  : libverto-libevent-0.2.5-4.el7.x86_64                                                                                                                                                                                                                      2/15 
      Verifying  : 1:quota-4.01-19.el7.x86_64                                                                                                                                                                                                                                3/15 
      Verifying  : gssproxy-0.7.0-26.el7.x86_64                                                                                                                                                                                                                              4/15 
      Verifying  : libpath_utils-0.2.1-32.el7.x86_64                                                                                                                                                                                                                         5/15 
      Verifying  : libnfsidmap-0.25-19.el7.x86_64                                                                                                                                                                                                                            6/15 
      Verifying  : keyutils-1.5.8-3.el7.x86_64                                                                                                                                                                                                                               7/15 
      Verifying  : 1:nfs-utils-1.3.0-0.65.el7.x86_64                                                                                                                                                                                                                         8/15 
      Verifying  : tcp_wrappers-7.6-77.el7.x86_64                                                                                                                                                                                                                            9/15 
      Verifying  : libcollection-0.7.0-32.el7.x86_64                                                                                                                                                                                                                        10/15 
      Verifying  : libref_array-0.1.5-32.el7.x86_64                                                                                                                                                                                                                         11/15 
      Verifying  : libbasicobjects-0.1.1-32.el7.x86_64                                                                                                                                                                                                                      12/15 
      Verifying  : rpcbind-0.2.0-48.el7.x86_64                                                                                                                                                                                                                              13/15 
      Verifying  : libini_config-1.3.1-32.el7.x86_64                                                                                                                                                                                                                        14/15 
      Verifying  : 1:quota-nls-4.01-19.el7.noarch                                                                                                                                                                                                                           15/15 
    
    Installed:
      nfs-utils.x86_64 1:1.3.0-0.65.el7                                                                                                                                                                                                                                           
    
    Dependency Installed:
      gssproxy.x86_64 0:0.7.0-26.el7        keyutils.x86_64 0:1.5.8-3.el7       libbasicobjects.x86_64 0:0.1.1-32.el7     libcollection.x86_64 0:0.7.0-32.el7    libini_config.x86_64 0:1.3.1-32.el7    libnfsidmap.x86_64 0:0.25-19.el7    libpath_utils.x86_64 0:0.2.1-32.el7   
      libref_array.x86_64 0:0.1.5-32.el7    libtirpc.x86_64 0:0.2.4-0.16.el7    libverto-libevent.x86_64 0:0.2.5-4.el7    quota.x86_64 1:4.01-19.el7             quota-nls.noarch 1:4.01-19.el7         rpcbind.x86_64 0:0.2.0-48.el7       tcp_wrappers.x86_64 0:7.6-77.el7      
    
    Complete!
    [root@test110.yinzhengjie.org.cn ~]# 
    [root@test110.yinzhengjie.org.cn ~]# yum -y install nfs-utils
    [root@test110.yinzhengjie.org.cn ~]# mkdir -pv /yinzhengjie/data/volume{1,2,3,4,5}
    mkdir: created directory ‘/yinzhengjie/data’
    mkdir: created directory ‘/yinzhengjie/data/volume1’
    mkdir: created directory ‘/yinzhengjie/data/volume2’
    mkdir: created directory ‘/yinzhengjie/data/volume3’
    mkdir: created directory ‘/yinzhengjie/data/volume4’
    mkdir: created directory ‘/yinzhengjie/data/volume5’
    [root@test110.yinzhengjie.org.cn ~]# 
    [root@test110.yinzhengjie.org.cn ~]# 
    [root@test110.yinzhengjie.org.cn ~]# ll /yinzhengjie/data/ -R
    /yinzhengjie/data/:
    total 0
    drwxr-xr-x 2 root root 6 Feb 10 06:04 volume1
    drwxr-xr-x 2 root root 6 Feb 10 06:04 volume2
    drwxr-xr-x 2 root root 6 Feb 10 06:04 volume3
    drwxr-xr-x 2 root root 6 Feb 10 06:04 volume4
    drwxr-xr-x 2 root root 6 Feb 10 06:04 volume5
    
    /yinzhengjie/data/volume1:
    total 0
    
    /yinzhengjie/data/volume2:
    total 0
    
    /yinzhengjie/data/volume3:
    total 0
    
    /yinzhengjie/data/volume4:
    total 0
    
    /yinzhengjie/data/volume5:
    total 0
    [root@test110.yinzhengjie.org.cn ~]# 
    [root@test110.yinzhengjie.org.cn ~]# mkdir -pv /yinzhengjie/data/volume{1,2,3,4,5}
    [root@test110.yinzhengjie.org.cn ~]# vim /etc/exports
    [root@test110.yinzhengjie.org.cn ~]# 
    [root@test110.yinzhengjie.org.cn ~]# cat /etc/exports
    /yinzhengjie/data/volume1    172.200.0.0/21(rw,no_root_squash)
    [root@test110.yinzhengjie.org.cn ~]# 
    [root@test110.yinzhengjie.org.cn ~]# vim /etc/exports
    [root@test110.yinzhengjie.org.cn ~]# systemctl status nfs
    ● nfs-server.service - NFS server and services
       Loaded: loaded (/usr/lib/systemd/system/nfs-server.service; disabled; vendor preset: disabled)
       Active: inactive (dead)
    [root@test110.yinzhengjie.org.cn ~]# 
    [root@test110.yinzhengjie.org.cn ~]# systemctl start nfs
    [root@test110.yinzhengjie.org.cn ~]# 
    [root@test110.yinzhengjie.org.cn ~]# systemctl enable nfs
    Created symlink from /etc/systemd/system/multi-user.target.wants/nfs-server.service to /usr/lib/systemd/system/nfs-server.service.
    [root@test110.yinzhengjie.org.cn ~]# 
    [root@test110.yinzhengjie.org.cn ~]# systemctl status nfs
    ● nfs-server.service - NFS server and services
       Loaded: loaded (/usr/lib/systemd/system/nfs-server.service; enabled; vendor preset: disabled)
      Drop-In: /run/systemd/generator/nfs-server.service.d
               └─order-with-mounts.conf
       Active: active (exited) since Mon 2020-02-10 06:08:52 CST; 6s ago
     Main PID: 6116 (code=exited, status=0/SUCCESS)
       CGroup: /system.slice/nfs-server.service
    
    Feb 10 06:08:52 test110.yinzhengjie.org.cn systemd[1]: Starting NFS server and services...
    Feb 10 06:08:52 test110.yinzhengjie.org.cn systemd[1]: Started NFS server and services.
    [root@test110.yinzhengjie.org.cn ~]# 
    [root@test110.yinzhengjie.org.cn ~]# ss -ntl | grep 2049
    LISTEN     0      64           *:2049                     *:*                  
    LISTEN     0      64          :::2049                    :::*                  
    [root@test110.yinzhengjie.org.cn ~]# 
    [root@test110.yinzhengjie.org.cn ~]# iptables -vnL
    Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
     pkts bytes target     prot opt in     out     source               destination         
    
    Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
     pkts bytes target     prot opt in     out     source               destination         
    
    Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
     pkts bytes target     prot opt in     out     source               destination         
    [root@test110.yinzhengjie.org.cn ~]# 
    [root@test110.yinzhengjie.org.cn ~]# 
    [root@test110.yinzhengjie.org.cn ~]# iptables -vnL -t nat
    Chain PREROUTING (policy ACCEPT 0 packets, 0 bytes)
     pkts bytes target     prot opt in     out     source               destination         
    
    Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
     pkts bytes target     prot opt in     out     source               destination         
    
    Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
     pkts bytes target     prot opt in     out     source               destination         
    
    Chain POSTROUTING (policy ACCEPT 0 packets, 0 bytes)
     pkts bytes target     prot opt in     out     source               destination         
    [root@test110.yinzhengjie.org.cn ~]# 
    [root@test110.yinzhengjie.org.cn ~]# systemctl start nfs

    2>.在K8S集群的每一个节点安装nfs客户端驱动

    [root@master200.yinzhengjie.org.cn ~]# yum -y install nfs-utils
    Loaded plugins: fastestmirror
    Loading mirror speeds from cached hostfile
     * base: mirror.bit.edu.cn
     * extras: mirror.bit.edu.cn
     * updates: mirror.bit.edu.cn
    base                                                                                                                                                                                                                                                   | 3.6 kB  00:00:00     
    docker-ce-stable                                                                                                                                                                                                                                       | 3.5 kB  00:00:00     
    extras                                                                                                                                                                                                                                                 | 2.9 kB  00:00:00     
    kubernetes                                                                                                                                                                                                                                             | 1.4 kB  00:00:00     
    updates                                                                                                                                                                                                                                                | 2.9 kB  00:00:00     
    Resolving Dependencies
    --> Running transaction check
    ---> Package nfs-utils.x86_64 1:1.3.0-0.65.el7 will be installed
    --> Processing Dependency: libtirpc >= 0.2.4-0.7 for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
    --> Processing Dependency: gssproxy >= 0.7.0-3 for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
    --> Processing Dependency: rpcbind for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
    --> Processing Dependency: quota for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
    --> Processing Dependency: libnfsidmap for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
    --> Processing Dependency: keyutils for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
    --> Processing Dependency: libtirpc.so.1()(64bit) for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
    --> Processing Dependency: libnfsidmap.so.0()(64bit) for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
    --> Running transaction check
    ---> Package gssproxy.x86_64 0:0.7.0-26.el7 will be installed
    --> Processing Dependency: libini_config >= 1.3.1-31 for package: gssproxy-0.7.0-26.el7.x86_64
    --> Processing Dependency: libverto-module-base for package: gssproxy-0.7.0-26.el7.x86_64
    --> Processing Dependency: libref_array.so.1(REF_ARRAY_0.1.1)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
    --> Processing Dependency: libini_config.so.3(INI_CONFIG_1.2.0)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
    --> Processing Dependency: libini_config.so.3(INI_CONFIG_1.1.0)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
    --> Processing Dependency: libref_array.so.1()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
    --> Processing Dependency: libini_config.so.3()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
    --> Processing Dependency: libcollection.so.2()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
    --> Processing Dependency: libbasicobjects.so.0()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
    ---> Package keyutils.x86_64 0:1.5.8-3.el7 will be installed
    ---> Package libnfsidmap.x86_64 0:0.25-19.el7 will be installed
    ---> Package libtirpc.x86_64 0:0.2.4-0.16.el7 will be installed
    ---> Package quota.x86_64 1:4.01-19.el7 will be installed
    --> Processing Dependency: quota-nls = 1:4.01-19.el7 for package: 1:quota-4.01-19.el7.x86_64
    --> Processing Dependency: tcp_wrappers for package: 1:quota-4.01-19.el7.x86_64
    ---> Package rpcbind.x86_64 0:0.2.0-48.el7 will be installed
    --> Running transaction check
    ---> Package libbasicobjects.x86_64 0:0.1.1-32.el7 will be installed
    ---> Package libcollection.x86_64 0:0.7.0-32.el7 will be installed
    ---> Package libini_config.x86_64 0:1.3.1-32.el7 will be installed
    --> Processing Dependency: libpath_utils.so.1(PATH_UTILS_0.2.1)(64bit) for package: libini_config-1.3.1-32.el7.x86_64
    --> Processing Dependency: libpath_utils.so.1()(64bit) for package: libini_config-1.3.1-32.el7.x86_64
    ---> Package libref_array.x86_64 0:0.1.5-32.el7 will be installed
    ---> Package libverto-libevent.x86_64 0:0.2.5-4.el7 will be installed
    ---> Package quota-nls.noarch 1:4.01-19.el7 will be installed
    ---> Package tcp_wrappers.x86_64 0:7.6-77.el7 will be installed
    --> Running transaction check
    ---> Package libpath_utils.x86_64 0:0.2.1-32.el7 will be installed
    --> Finished Dependency Resolution
    
    Dependencies Resolved
    
    ==============================================================================================================================================================================================================================================================================
     Package                                                                 Arch                                                         Version                                                                Repository                                                  Size
    ==============================================================================================================================================================================================================================================================================
    Installing:
     nfs-utils                                                               x86_64                                                       1:1.3.0-0.65.el7                                                       base                                                       412 k
    Installing for dependencies:
     gssproxy                                                                x86_64                                                       0.7.0-26.el7                                                           base                                                       110 k
     keyutils                                                                x86_64                                                       1.5.8-3.el7                                                            base                                                        54 k
     libbasicobjects                                                         x86_64                                                       0.1.1-32.el7                                                           base                                                        26 k
     libcollection                                                           x86_64                                                       0.7.0-32.el7                                                           base                                                        42 k
     libini_config                                                           x86_64                                                       1.3.1-32.el7                                                           base                                                        64 k
     libnfsidmap                                                             x86_64                                                       0.25-19.el7                                                            base                                                        50 k
     libpath_utils                                                           x86_64                                                       0.2.1-32.el7                                                           base                                                        28 k
     libref_array                                                            x86_64                                                       0.1.5-32.el7                                                           base                                                        27 k
     libtirpc                                                                x86_64                                                       0.2.4-0.16.el7                                                         base                                                        89 k
     libverto-libevent                                                       x86_64                                                       0.2.5-4.el7                                                            base                                                       8.9 k
     quota                                                                   x86_64                                                       1:4.01-19.el7                                                          base                                                       179 k
     quota-nls                                                               noarch                                                       1:4.01-19.el7                                                          base                                                        90 k
     rpcbind                                                                 x86_64                                                       0.2.0-48.el7                                                           base                                                        60 k
     tcp_wrappers                                                            x86_64                                                       7.6-77.el7                                                             base                                                        78 k
    
    Transaction Summary
    ==============================================================================================================================================================================================================================================================================
    Install  1 Package (+14 Dependent packages)
    
    Total download size: 1.3 M
    Installed size: 3.6 M
    Downloading packages:
    (1/15): keyutils-1.5.8-3.el7.x86_64.rpm                                                                                                                                                                                                                |  54 kB  00:00:00     
    (2/15): gssproxy-0.7.0-26.el7.x86_64.rpm                                                                                                                                                                                                               | 110 kB  00:00:00     
    (3/15): libini_config-1.3.1-32.el7.x86_64.rpm                                                                                                                                                                                                          |  64 kB  00:00:00     
    (4/15): libbasicobjects-0.1.1-32.el7.x86_64.rpm                                                                                                                                                                                                        |  26 kB  00:00:00     
    (5/15): libpath_utils-0.2.1-32.el7.x86_64.rpm                                                                                                                                                                                                          |  28 kB  00:00:00     
    (6/15): libcollection-0.7.0-32.el7.x86_64.rpm                                                                                                                                                                                                          |  42 kB  00:00:00     
    (7/15): libref_array-0.1.5-32.el7.x86_64.rpm                                                                                                                                                                                                           |  27 kB  00:00:00     
    (8/15): libtirpc-0.2.4-0.16.el7.x86_64.rpm                                                                                                                                                                                                             |  89 kB  00:00:00     
    (9/15): libverto-libevent-0.2.5-4.el7.x86_64.rpm                                                                                                                                                                                                       | 8.9 kB  00:00:00     
    (10/15): quota-4.01-19.el7.x86_64.rpm                                                                                                                                                                                                                  | 179 kB  00:00:00     
    (11/15): libnfsidmap-0.25-19.el7.x86_64.rpm                                                                                                                                                                                                            |  50 kB  00:00:00     
    (12/15): rpcbind-0.2.0-48.el7.x86_64.rpm                                                                                                                                                                                                               |  60 kB  00:00:00     
    (13/15): nfs-utils-1.3.0-0.65.el7.x86_64.rpm                                                                                                                                                                                                           | 412 kB  00:00:00     
    (14/15): tcp_wrappers-7.6-77.el7.x86_64.rpm                                                                                                                                                                                                            |  78 kB  00:00:00     
    (15/15): quota-nls-4.01-19.el7.noarch.rpm                                                                                                                                                                                                              |  90 kB  00:00:00     
    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    Total                                                                                                                                                                                                                                         1.4 MB/s | 1.3 MB  00:00:00     
    Running transaction check
    Running transaction test
    Transaction test succeeded
    Running transaction
      Installing : libbasicobjects-0.1.1-32.el7.x86_64                                                                                                                                                                                                                       1/15 
      Installing : libref_array-0.1.5-32.el7.x86_64                                                                                                                                                                                                                          2/15 
      Installing : libcollection-0.7.0-32.el7.x86_64                                                                                                                                                                                                                         3/15 
      Installing : libtirpc-0.2.4-0.16.el7.x86_64                                                                                                                                                                                                                            4/15 
      Installing : rpcbind-0.2.0-48.el7.x86_64                                                                                                                                                                                                                               5/15 
      Installing : 1:quota-nls-4.01-19.el7.noarch                                                                                                                                                                                                                            6/15 
      Installing : tcp_wrappers-7.6-77.el7.x86_64                                                                                                                                                                                                                            7/15 
      Installing : 1:quota-4.01-19.el7.x86_64                                                                                                                                                                                                                                8/15 
      Installing : keyutils-1.5.8-3.el7.x86_64                                                                                                                                                                                                                               9/15 
      Installing : libnfsidmap-0.25-19.el7.x86_64                                                                                                                                                                                                                           10/15 
      Installing : libpath_utils-0.2.1-32.el7.x86_64                                                                                                                                                                                                                        11/15 
      Installing : libini_config-1.3.1-32.el7.x86_64                                                                                                                                                                                                                        12/15 
      Installing : libverto-libevent-0.2.5-4.el7.x86_64                                                                                                                                                                                                                     13/15 
      Installing : gssproxy-0.7.0-26.el7.x86_64                                                                                                                                                                                                                             14/15 
      Installing : 1:nfs-utils-1.3.0-0.65.el7.x86_64                                                                                                                                                                                                                        15/15 
      Verifying  : libtirpc-0.2.4-0.16.el7.x86_64                                                                                                                                                                                                                            1/15 
      Verifying  : libverto-libevent-0.2.5-4.el7.x86_64                                                                                                                                                                                                                      2/15 
      Verifying  : 1:quota-4.01-19.el7.x86_64                                                                                                                                                                                                                                3/15 
      Verifying  : gssproxy-0.7.0-26.el7.x86_64                                                                                                                                                                                                                              4/15 
      Verifying  : libpath_utils-0.2.1-32.el7.x86_64                                                                                                                                                                                                                         5/15 
      Verifying  : libnfsidmap-0.25-19.el7.x86_64                                                                                                                                                                                                                            6/15 
      Verifying  : keyutils-1.5.8-3.el7.x86_64                                                                                                                                                                                                                               7/15 
      Verifying  : 1:nfs-utils-1.3.0-0.65.el7.x86_64                                                                                                                                                                                                                         8/15 
      Verifying  : tcp_wrappers-7.6-77.el7.x86_64                                                                                                                                                                                                                            9/15 
      Verifying  : libcollection-0.7.0-32.el7.x86_64                                                                                                                                                                                                                        10/15 
      Verifying  : libref_array-0.1.5-32.el7.x86_64                                                                                                                                                                                                                         11/15 
      Verifying  : libbasicobjects-0.1.1-32.el7.x86_64                                                                                                                                                                                                                      12/15 
      Verifying  : rpcbind-0.2.0-48.el7.x86_64                                                                                                                                                                                                                              13/15 
      Verifying  : libini_config-1.3.1-32.el7.x86_64                                                                                                                                                                                                                        14/15 
      Verifying  : 1:quota-nls-4.01-19.el7.noarch                                                                                                                                                                                                                           15/15 
    
    Installed:
      nfs-utils.x86_64 1:1.3.0-0.65.el7                                                                                                                                                                                                                                           
    
    Dependency Installed:
      gssproxy.x86_64 0:0.7.0-26.el7        keyutils.x86_64 0:1.5.8-3.el7       libbasicobjects.x86_64 0:0.1.1-32.el7     libcollection.x86_64 0:0.7.0-32.el7    libini_config.x86_64 0:1.3.1-32.el7    libnfsidmap.x86_64 0:0.25-19.el7    libpath_utils.x86_64 0:0.2.1-32.el7   
      libref_array.x86_64 0:0.1.5-32.el7    libtirpc.x86_64 0:0.2.4-0.16.el7    libverto-libevent.x86_64 0:0.2.5-4.el7    quota.x86_64 1:4.01-19.el7             quota-nls.noarch 1:4.01-19.el7         rpcbind.x86_64 0:0.2.0-48.el7       tcp_wrappers.x86_64 0:7.6-77.el7      
    
    Complete!
    [root@master200.yinzhengjie.org.cn ~]# 
    [root@master200.yinzhengjie.org.cn ~]# yum -y install nfs-utils
    [root@node201.yinzhengjie.org.cn ~]#  yum -y install nfs-utils
    Loaded plugins: fastestmirror
    Loading mirror speeds from cached hostfile
     * base: mirrors.aliyun.com
     * extras: mirrors.aliyun.com
     * updates: mirror.bit.edu.cn
    base                                                                                                                                                                                                                                                   | 3.6 kB  00:00:00     
    docker-ce-stable                                                                                                                                                                                                                                       | 3.5 kB  00:00:00     
    extras                                                                                                                                                                                                                                                 | 2.9 kB  00:00:00     
    kubernetes                                                                                                                                                                                                                                             | 1.4 kB  00:00:00     
    updates                                                                                                                                                                                                                                                | 2.9 kB  00:00:00     
    updates/7/x86_64/primary_db                                                                                                                                                                                                                            | 6.7 MB  00:00:00     
    Resolving Dependencies
    --> Running transaction check
    ---> Package nfs-utils.x86_64 1:1.3.0-0.65.el7 will be installed
    --> Processing Dependency: libtirpc >= 0.2.4-0.7 for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
    --> Processing Dependency: gssproxy >= 0.7.0-3 for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
    --> Processing Dependency: rpcbind for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
    --> Processing Dependency: quota for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
    --> Processing Dependency: libnfsidmap for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
    --> Processing Dependency: keyutils for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
    --> Processing Dependency: libtirpc.so.1()(64bit) for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
    --> Processing Dependency: libnfsidmap.so.0()(64bit) for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
    --> Running transaction check
    ---> Package gssproxy.x86_64 0:0.7.0-26.el7 will be installed
    --> Processing Dependency: libini_config >= 1.3.1-31 for package: gssproxy-0.7.0-26.el7.x86_64
    --> Processing Dependency: libverto-module-base for package: gssproxy-0.7.0-26.el7.x86_64
    --> Processing Dependency: libref_array.so.1(REF_ARRAY_0.1.1)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
    --> Processing Dependency: libini_config.so.3(INI_CONFIG_1.2.0)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
    --> Processing Dependency: libini_config.so.3(INI_CONFIG_1.1.0)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
    --> Processing Dependency: libref_array.so.1()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
    --> Processing Dependency: libini_config.so.3()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
    --> Processing Dependency: libcollection.so.2()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
    --> Processing Dependency: libbasicobjects.so.0()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
    ---> Package keyutils.x86_64 0:1.5.8-3.el7 will be installed
    ---> Package libnfsidmap.x86_64 0:0.25-19.el7 will be installed
    ---> Package libtirpc.x86_64 0:0.2.4-0.16.el7 will be installed
    ---> Package quota.x86_64 1:4.01-19.el7 will be installed
    --> Processing Dependency: quota-nls = 1:4.01-19.el7 for package: 1:quota-4.01-19.el7.x86_64
    --> Processing Dependency: tcp_wrappers for package: 1:quota-4.01-19.el7.x86_64
    ---> Package rpcbind.x86_64 0:0.2.0-48.el7 will be installed
    --> Running transaction check
    ---> Package libbasicobjects.x86_64 0:0.1.1-32.el7 will be installed
    ---> Package libcollection.x86_64 0:0.7.0-32.el7 will be installed
    ---> Package libini_config.x86_64 0:1.3.1-32.el7 will be installed
    --> Processing Dependency: libpath_utils.so.1(PATH_UTILS_0.2.1)(64bit) for package: libini_config-1.3.1-32.el7.x86_64
    --> Processing Dependency: libpath_utils.so.1()(64bit) for package: libini_config-1.3.1-32.el7.x86_64
    ---> Package libref_array.x86_64 0:0.1.5-32.el7 will be installed
    ---> Package libverto-libevent.x86_64 0:0.2.5-4.el7 will be installed
    ---> Package quota-nls.noarch 1:4.01-19.el7 will be installed
    ---> Package tcp_wrappers.x86_64 0:7.6-77.el7 will be installed
    --> Running transaction check
    ---> Package libpath_utils.x86_64 0:0.2.1-32.el7 will be installed
    --> Finished Dependency Resolution
    
    Dependencies Resolved
    
    ==============================================================================================================================================================================================================================================================================
     Package                                                                 Arch                                                         Version                                                                Repository                                                  Size
    ==============================================================================================================================================================================================================================================================================
    Installing:
     nfs-utils                                                               x86_64                                                       1:1.3.0-0.65.el7                                                       base                                                       412 k
    Installing for dependencies:
     gssproxy                                                                x86_64                                                       0.7.0-26.el7                                                           base                                                       110 k
     keyutils                                                                x86_64                                                       1.5.8-3.el7                                                            base                                                        54 k
     libbasicobjects                                                         x86_64                                                       0.1.1-32.el7                                                           base                                                        26 k
     libcollection                                                           x86_64                                                       0.7.0-32.el7                                                           base                                                        42 k
     libini_config                                                           x86_64                                                       1.3.1-32.el7                                                           base                                                        64 k
     libnfsidmap                                                             x86_64                                                       0.25-19.el7                                                            base                                                        50 k
     libpath_utils                                                           x86_64                                                       0.2.1-32.el7                                                           base                                                        28 k
     libref_array                                                            x86_64                                                       0.1.5-32.el7                                                           base                                                        27 k
     libtirpc                                                                x86_64                                                       0.2.4-0.16.el7                                                         base                                                        89 k
     libverto-libevent                                                       x86_64                                                       0.2.5-4.el7                                                            base                                                       8.9 k
     quota                                                                   x86_64                                                       1:4.01-19.el7                                                          base                                                       179 k
     quota-nls                                                               noarch                                                       1:4.01-19.el7                                                          base                                                        90 k
     rpcbind                                                                 x86_64                                                       0.2.0-48.el7                                                           base                                                        60 k
     tcp_wrappers                                                            x86_64                                                       7.6-77.el7                                                             base                                                        78 k
    
    Transaction Summary
    ==============================================================================================================================================================================================================================================================================
    Install  1 Package (+14 Dependent packages)
    
    Total download size: 1.3 M
    Installed size: 3.6 M
    Downloading packages:
    (1/15): gssproxy-0.7.0-26.el7.x86_64.rpm                                                                                                                                                                                                               | 110 kB  00:00:00     
    (2/15): keyutils-1.5.8-3.el7.x86_64.rpm                                                                                                                                                                                                                |  54 kB  00:00:00     
    (3/15): libcollection-0.7.0-32.el7.x86_64.rpm                                                                                                                                                                                                          |  42 kB  00:00:00     
    (4/15): libnfsidmap-0.25-19.el7.x86_64.rpm                                                                                                                                                                                                             |  50 kB  00:00:00     
    (5/15): libbasicobjects-0.1.1-32.el7.x86_64.rpm                                                                                                                                                                                                        |  26 kB  00:00:00     
    (6/15): libini_config-1.3.1-32.el7.x86_64.rpm                                                                                                                                                                                                          |  64 kB  00:00:00     
    (7/15): libtirpc-0.2.4-0.16.el7.x86_64.rpm                                                                                                                                                                                                             |  89 kB  00:00:00     
    (8/15): libref_array-0.1.5-32.el7.x86_64.rpm                                                                                                                                                                                                           |  27 kB  00:00:00     
    (9/15): libverto-libevent-0.2.5-4.el7.x86_64.rpm                                                                                                                                                                                                       | 8.9 kB  00:00:00     
    (10/15): libpath_utils-0.2.1-32.el7.x86_64.rpm                                                                                                                                                                                                         |  28 kB  00:00:00     
    (11/15): nfs-utils-1.3.0-0.65.el7.x86_64.rpm                                                                                                                                                                                                           | 412 kB  00:00:00     
    (12/15): quota-4.01-19.el7.x86_64.rpm                                                                                                                                                                                                                  | 179 kB  00:00:00     
    (13/15): quota-nls-4.01-19.el7.noarch.rpm                                                                                                                                                                                                              |  90 kB  00:00:00     
    (14/15): tcp_wrappers-7.6-77.el7.x86_64.rpm                                                                                                                                                                                                            |  78 kB  00:00:00     
    (15/15): rpcbind-0.2.0-48.el7.x86_64.rpm                                                                                                                                                                                                               |  60 kB  00:00:00     
    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    Total                                                                                                                                                                                                                                         2.2 MB/s | 1.3 MB  00:00:00     
    Running transaction check
    Running transaction test
    Transaction test succeeded
    Running transaction
      Installing : libbasicobjects-0.1.1-32.el7.x86_64                                                                                                                                                                                                                       1/15 
      Installing : libref_array-0.1.5-32.el7.x86_64                                                                                                                                                                                                                          2/15 
      Installing : libcollection-0.7.0-32.el7.x86_64                                                                                                                                                                                                                         3/15 
      Installing : libtirpc-0.2.4-0.16.el7.x86_64                                                                                                                                                                                                                            4/15 
      Installing : rpcbind-0.2.0-48.el7.x86_64                                                                                                                                                                                                                               5/15 
      Installing : 1:quota-nls-4.01-19.el7.noarch                                                                                                                                                                                                                            6/15 
      Installing : tcp_wrappers-7.6-77.el7.x86_64                                                                                                                                                                                                                            7/15 
      Installing : 1:quota-4.01-19.el7.x86_64                                                                                                                                                                                                                                8/15 
      Installing : keyutils-1.5.8-3.el7.x86_64                                                                                                                                                                                                                               9/15 
      Installing : libnfsidmap-0.25-19.el7.x86_64                                                                                                                                                                                                                           10/15 
      Installing : libpath_utils-0.2.1-32.el7.x86_64                                                                                                                                                                                                                        11/15 
      Installing : libini_config-1.3.1-32.el7.x86_64                                                                                                                                                                                                                        12/15 
      Installing : libverto-libevent-0.2.5-4.el7.x86_64                                                                                                                                                                                                                     13/15 
      Installing : gssproxy-0.7.0-26.el7.x86_64                                                                                                                                                                                                                             14/15 
      Installing : 1:nfs-utils-1.3.0-0.65.el7.x86_64                                                                                                                                                                                                                        15/15 
      Verifying  : libtirpc-0.2.4-0.16.el7.x86_64                                                                                                                                                                                                                            1/15 
      Verifying  : libverto-libevent-0.2.5-4.el7.x86_64                                                                                                                                                                                                                      2/15 
      Verifying  : 1:quota-4.01-19.el7.x86_64                                                                                                                                                                                                                                3/15 
      Verifying  : gssproxy-0.7.0-26.el7.x86_64                                                                                                                                                                                                                              4/15 
      Verifying  : libpath_utils-0.2.1-32.el7.x86_64                                                                                                                                                                                                                         5/15 
      Verifying  : libnfsidmap-0.25-19.el7.x86_64                                                                                                                                                                                                                            6/15 
      Verifying  : keyutils-1.5.8-3.el7.x86_64                                                                                                                                                                                                                               7/15 
      Verifying  : 1:nfs-utils-1.3.0-0.65.el7.x86_64                                                                                                                                                                                                                         8/15 
      Verifying  : tcp_wrappers-7.6-77.el7.x86_64                                                                                                                                                                                                                            9/15 
      Verifying  : libcollection-0.7.0-32.el7.x86_64                                                                                                                                                                                                                        10/15 
      Verifying  : libref_array-0.1.5-32.el7.x86_64                                                                                                                                                                                                                         11/15 
      Verifying  : libbasicobjects-0.1.1-32.el7.x86_64                                                                                                                                                                                                                      12/15 
      Verifying  : rpcbind-0.2.0-48.el7.x86_64                                                                                                                                                                                                                              13/15 
      Verifying  : libini_config-1.3.1-32.el7.x86_64                                                                                                                                                                                                                        14/15 
      Verifying  : 1:quota-nls-4.01-19.el7.noarch                                                                                                                                                                                                                           15/15 
    
    Installed:
      nfs-utils.x86_64 1:1.3.0-0.65.el7                                                                                                                                                                                                                                           
    
    Dependency Installed:
      gssproxy.x86_64 0:0.7.0-26.el7        keyutils.x86_64 0:1.5.8-3.el7       libbasicobjects.x86_64 0:0.1.1-32.el7     libcollection.x86_64 0:0.7.0-32.el7    libini_config.x86_64 0:1.3.1-32.el7    libnfsidmap.x86_64 0:0.25-19.el7    libpath_utils.x86_64 0:0.2.1-32.el7   
      libref_array.x86_64 0:0.1.5-32.el7    libtirpc.x86_64 0:0.2.4-0.16.el7    libverto-libevent.x86_64 0:0.2.5-4.el7    quota.x86_64 1:4.01-19.el7             quota-nls.noarch 1:4.01-19.el7         rpcbind.x86_64 0:0.2.0-48.el7       tcp_wrappers.x86_64 0:7.6-77.el7      
    
    Complete!
    [root@node201.yinzhengjie.org.cn ~]# 
    [root@node201.yinzhengjie.org.cn ~]# yum -y install nfs-utils
    [root@node202.yinzhengjie.org.cn ~]# yum -y install nfs-utils
    Loaded plugins: fastestmirror
    Loading mirror speeds from cached hostfile
     * base: mirror.bit.edu.cn
     * extras: mirror.bit.edu.cn
     * updates: mirror.bit.edu.cn
    base                                                                                                                                                                                                                                                   | 3.6 kB  00:00:00     
    docker-ce-stable                                                                                                                                                                                                                                       | 3.5 kB  00:00:00     
    extras                                                                                                                                                                                                                                                 | 2.9 kB  00:00:00     
    kubernetes                                                                                                                                                                                                                                             | 1.4 kB  00:00:00     
    updates                                                                                                                                                                                                                                                | 2.9 kB  00:00:00     
    updates/7/x86_64/primary_db                                                                                                                                                                                                                            | 6.7 MB  00:00:01     
    Resolving Dependencies
    --> Running transaction check
    ---> Package nfs-utils.x86_64 1:1.3.0-0.65.el7 will be installed
    --> Processing Dependency: libtirpc >= 0.2.4-0.7 for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
    --> Processing Dependency: gssproxy >= 0.7.0-3 for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
    --> Processing Dependency: rpcbind for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
    --> Processing Dependency: quota for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
    --> Processing Dependency: libnfsidmap for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
    --> Processing Dependency: keyutils for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
    --> Processing Dependency: libtirpc.so.1()(64bit) for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
    --> Processing Dependency: libnfsidmap.so.0()(64bit) for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
    --> Running transaction check
    ---> Package gssproxy.x86_64 0:0.7.0-26.el7 will be installed
    --> Processing Dependency: libini_config >= 1.3.1-31 for package: gssproxy-0.7.0-26.el7.x86_64
    --> Processing Dependency: libverto-module-base for package: gssproxy-0.7.0-26.el7.x86_64
    --> Processing Dependency: libref_array.so.1(REF_ARRAY_0.1.1)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
    --> Processing Dependency: libini_config.so.3(INI_CONFIG_1.2.0)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
    --> Processing Dependency: libini_config.so.3(INI_CONFIG_1.1.0)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
    --> Processing Dependency: libref_array.so.1()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
    --> Processing Dependency: libini_config.so.3()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
    --> Processing Dependency: libcollection.so.2()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
    --> Processing Dependency: libbasicobjects.so.0()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
    ---> Package keyutils.x86_64 0:1.5.8-3.el7 will be installed
    ---> Package libnfsidmap.x86_64 0:0.25-19.el7 will be installed
    ---> Package libtirpc.x86_64 0:0.2.4-0.16.el7 will be installed
    ---> Package quota.x86_64 1:4.01-19.el7 will be installed
    --> Processing Dependency: quota-nls = 1:4.01-19.el7 for package: 1:quota-4.01-19.el7.x86_64
    --> Processing Dependency: tcp_wrappers for package: 1:quota-4.01-19.el7.x86_64
    ---> Package rpcbind.x86_64 0:0.2.0-48.el7 will be installed
    --> Running transaction check
    ---> Package libbasicobjects.x86_64 0:0.1.1-32.el7 will be installed
    ---> Package libcollection.x86_64 0:0.7.0-32.el7 will be installed
    ---> Package libini_config.x86_64 0:1.3.1-32.el7 will be installed
    --> Processing Dependency: libpath_utils.so.1(PATH_UTILS_0.2.1)(64bit) for package: libini_config-1.3.1-32.el7.x86_64
    --> Processing Dependency: libpath_utils.so.1()(64bit) for package: libini_config-1.3.1-32.el7.x86_64
    ---> Package libref_array.x86_64 0:0.1.5-32.el7 will be installed
    ---> Package libverto-libevent.x86_64 0:0.2.5-4.el7 will be installed
    ---> Package quota-nls.noarch 1:4.01-19.el7 will be installed
    ---> Package tcp_wrappers.x86_64 0:7.6-77.el7 will be installed
    --> Running transaction check
    ---> Package libpath_utils.x86_64 0:0.2.1-32.el7 will be installed
    --> Finished Dependency Resolution
    
    Dependencies Resolved
    
    ==============================================================================================================================================================================================================================================================================
     Package                                                                 Arch                                                         Version                                                                Repository                                                  Size
    ==============================================================================================================================================================================================================================================================================
    Installing:
     nfs-utils                                                               x86_64                                                       1:1.3.0-0.65.el7                                                       base                                                       412 k
    Installing for dependencies:
     gssproxy                                                                x86_64                                                       0.7.0-26.el7                                                           base                                                       110 k
     keyutils                                                                x86_64                                                       1.5.8-3.el7                                                            base                                                        54 k
     libbasicobjects                                                         x86_64                                                       0.1.1-32.el7                                                           base                                                        26 k
     libcollection                                                           x86_64                                                       0.7.0-32.el7                                                           base                                                        42 k
     libini_config                                                           x86_64                                                       1.3.1-32.el7                                                           base                                                        64 k
     libnfsidmap                                                             x86_64                                                       0.25-19.el7                                                            base                                                        50 k
     libpath_utils                                                           x86_64                                                       0.2.1-32.el7                                                           base                                                        28 k
     libref_array                                                            x86_64                                                       0.1.5-32.el7                                                           base                                                        27 k
     libtirpc                                                                x86_64                                                       0.2.4-0.16.el7                                                         base                                                        89 k
     libverto-libevent                                                       x86_64                                                       0.2.5-4.el7                                                            base                                                       8.9 k
     quota                                                                   x86_64                                                       1:4.01-19.el7                                                          base                                                       179 k
     quota-nls                                                               noarch                                                       1:4.01-19.el7                                                          base                                                        90 k
     rpcbind                                                                 x86_64                                                       0.2.0-48.el7                                                           base                                                        60 k
     tcp_wrappers                                                            x86_64                                                       7.6-77.el7                                                             base                                                        78 k
    
    Transaction Summary
    ==============================================================================================================================================================================================================================================================================
    Install  1 Package (+14 Dependent packages)
    
    Total download size: 1.3 M
    Installed size: 3.6 M
    Downloading packages:
    (1/15): keyutils-1.5.8-3.el7.x86_64.rpm                                                                                                                                                                                                                |  54 kB  00:00:00     
    (2/15): libbasicobjects-0.1.1-32.el7.x86_64.rpm                                                                                                                                                                                                        |  26 kB  00:00:00     
    (3/15): libnfsidmap-0.25-19.el7.x86_64.rpm                                                                                                                                                                                                             |  50 kB  00:00:00     
    (4/15): libini_config-1.3.1-32.el7.x86_64.rpm                                                                                                                                                                                                          |  64 kB  00:00:00     
    (5/15): libref_array-0.1.5-32.el7.x86_64.rpm                                                                                                                                                                                                           |  27 kB  00:00:00     
    (6/15): libpath_utils-0.2.1-32.el7.x86_64.rpm                                                                                                                                                                                                          |  28 kB  00:00:00     
    (7/15): libverto-libevent-0.2.5-4.el7.x86_64.rpm                                                                                                                                                                                                       | 8.9 kB  00:00:00     
    (8/15): gssproxy-0.7.0-26.el7.x86_64.rpm                                                                                                                                                                                                               | 110 kB  00:00:00     
    (9/15): libcollection-0.7.0-32.el7.x86_64.rpm                                                                                                                                                                                                          |  42 kB  00:00:00     
    (10/15): nfs-utils-1.3.0-0.65.el7.x86_64.rpm                                                                                                                                                                                                           | 412 kB  00:00:00     
    (11/15): libtirpc-0.2.4-0.16.el7.x86_64.rpm                                                                                                                                                                                                            |  89 kB  00:00:00     
    (12/15): quota-4.01-19.el7.x86_64.rpm                                                                                                                                                                                                                  | 179 kB  00:00:00     
    (13/15): tcp_wrappers-7.6-77.el7.x86_64.rpm                                                                                                                                                                                                            |  78 kB  00:00:00     
    (14/15): quota-nls-4.01-19.el7.noarch.rpm                                                                                                                                                                                                              |  90 kB  00:00:00     
    (15/15): rpcbind-0.2.0-48.el7.x86_64.rpm                                                                                                                                                                                                               |  60 kB  00:00:00     
    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    Total                                                                                                                                                                                                                                         1.9 MB/s | 1.3 MB  00:00:00     
    Running transaction check
    Running transaction test
    Transaction test succeeded
    Running transaction
      Installing : libbasicobjects-0.1.1-32.el7.x86_64                                                                                                                                                                                                                       1/15 
      Installing : libref_array-0.1.5-32.el7.x86_64                                                                                                                                                                                                                          2/15 
      Installing : libcollection-0.7.0-32.el7.x86_64                                                                                                                                                                                                                         3/15 
      Installing : libtirpc-0.2.4-0.16.el7.x86_64                                                                                                                                                                                                                            4/15 
      Installing : rpcbind-0.2.0-48.el7.x86_64                                                                                                                                                                                                                               5/15 
      Installing : 1:quota-nls-4.01-19.el7.noarch                                                                                                                                                                                                                            6/15 
      Installing : tcp_wrappers-7.6-77.el7.x86_64                                                                                                                                                                                                                            7/15 
      Installing : 1:quota-4.01-19.el7.x86_64                                                                                                                                                                                                                                8/15 
      Installing : keyutils-1.5.8-3.el7.x86_64                                                                                                                                                                                                                               9/15 
      Installing : libnfsidmap-0.25-19.el7.x86_64                                                                                                                                                                                                                           10/15 
      Installing : libpath_utils-0.2.1-32.el7.x86_64                                                                                                                                                                                                                        11/15 
      Installing : libini_config-1.3.1-32.el7.x86_64                                                                                                                                                                                                                        12/15 
      Installing : libverto-libevent-0.2.5-4.el7.x86_64                                                                                                                                                                                                                     13/15 
      Installing : gssproxy-0.7.0-26.el7.x86_64                                                                                                                                                                                                                             14/15 
      Installing : 1:nfs-utils-1.3.0-0.65.el7.x86_64                                                                                                                                                                                                                        15/15 
      Verifying  : libtirpc-0.2.4-0.16.el7.x86_64                                                                                                                                                                                                                            1/15 
      Verifying  : libverto-libevent-0.2.5-4.el7.x86_64                                                                                                                                                                                                                      2/15 
      Verifying  : 1:quota-4.01-19.el7.x86_64                                                                                                                                                                                                                                3/15 
      Verifying  : gssproxy-0.7.0-26.el7.x86_64                                                                                                                                                                                                                              4/15 
      Verifying  : libpath_utils-0.2.1-32.el7.x86_64                                                                                                                                                                                                                         5/15 
      Verifying  : libnfsidmap-0.25-19.el7.x86_64                                                                                                                                                                                                                            6/15 
      Verifying  : keyutils-1.5.8-3.el7.x86_64                                                                                                                                                                                                                               7/15 
      Verifying  : 1:nfs-utils-1.3.0-0.65.el7.x86_64                                                                                                                                                                                                                         8/15 
      Verifying  : tcp_wrappers-7.6-77.el7.x86_64                                                                                                                                                                                                                            9/15 
      Verifying  : libcollection-0.7.0-32.el7.x86_64                                                                                                                                                                                                                        10/15 
      Verifying  : libref_array-0.1.5-32.el7.x86_64                                                                                                                                                                                                                         11/15 
      Verifying  : libbasicobjects-0.1.1-32.el7.x86_64                                                                                                                                                                                                                      12/15 
      Verifying  : rpcbind-0.2.0-48.el7.x86_64                                                                                                                                                                                                                              13/15 
      Verifying  : libini_config-1.3.1-32.el7.x86_64                                                                                                                                                                                                                        14/15 
      Verifying  : 1:quota-nls-4.01-19.el7.noarch                                                                                                                                                                                                                           15/15 
    
    Installed:
      nfs-utils.x86_64 1:1.3.0-0.65.el7                                                                                                                                                                                                                                           
    
    Dependency Installed:
      gssproxy.x86_64 0:0.7.0-26.el7        keyutils.x86_64 0:1.5.8-3.el7       libbasicobjects.x86_64 0:0.1.1-32.el7     libcollection.x86_64 0:0.7.0-32.el7    libini_config.x86_64 0:1.3.1-32.el7    libnfsidmap.x86_64 0:0.25-19.el7    libpath_utils.x86_64 0:0.2.1-32.el7   
      libref_array.x86_64 0:0.1.5-32.el7    libtirpc.x86_64 0:0.2.4-0.16.el7    libverto-libevent.x86_64 0:0.2.5-4.el7    quota.x86_64 1:4.01-19.el7             quota-nls.noarch 1:4.01-19.el7         rpcbind.x86_64 0:0.2.0-48.el7       tcp_wrappers.x86_64 0:7.6-77.el7      
    
    Complete!
    [root@node202.yinzhengjie.org.cn ~]# 
    [root@node202.yinzhengjie.org.cn ~]# yum -y install nfs-utils
    [root@node203.yinzhengjie.org.cn ~]# yum -y install nfs-utils
    Loaded plugins: fastestmirror
    Loading mirror speeds from cached hostfile
     * base: mirrors.aliyun.com
     * extras: mirrors.aliyun.com
     * updates: mirror.bit.edu.cn
    base                                                                                                                                                                                                                                                   | 3.6 kB  00:00:00     
    docker-ce-stable                                                                                                                                                                                                                                       | 3.5 kB  00:00:00     
    extras                                                                                                                                                                                                                                                 | 2.9 kB  00:00:00     
    kubernetes                                                                                                                                                                                                                                             | 1.4 kB  00:00:00     
    updates                                                                                                                                                                                                                                                | 2.9 kB  00:00:00     
    updates/7/x86_64/primary_db                                                                                                                                                                                                                            | 6.7 MB  00:00:00     
    Resolving Dependencies
    --> Running transaction check
    ---> Package nfs-utils.x86_64 1:1.3.0-0.65.el7 will be installed
    --> Processing Dependency: libtirpc >= 0.2.4-0.7 for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
    --> Processing Dependency: gssproxy >= 0.7.0-3 for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
    --> Processing Dependency: rpcbind for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
    --> Processing Dependency: quota for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
    --> Processing Dependency: libnfsidmap for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
    --> Processing Dependency: keyutils for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
    --> Processing Dependency: libtirpc.so.1()(64bit) for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
    --> Processing Dependency: libnfsidmap.so.0()(64bit) for package: 1:nfs-utils-1.3.0-0.65.el7.x86_64
    --> Running transaction check
    ---> Package gssproxy.x86_64 0:0.7.0-26.el7 will be installed
    --> Processing Dependency: libini_config >= 1.3.1-31 for package: gssproxy-0.7.0-26.el7.x86_64
    --> Processing Dependency: libverto-module-base for package: gssproxy-0.7.0-26.el7.x86_64
    --> Processing Dependency: libref_array.so.1(REF_ARRAY_0.1.1)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
    --> Processing Dependency: libini_config.so.3(INI_CONFIG_1.2.0)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
    --> Processing Dependency: libini_config.so.3(INI_CONFIG_1.1.0)(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
    --> Processing Dependency: libref_array.so.1()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
    --> Processing Dependency: libini_config.so.3()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
    --> Processing Dependency: libcollection.so.2()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
    --> Processing Dependency: libbasicobjects.so.0()(64bit) for package: gssproxy-0.7.0-26.el7.x86_64
    ---> Package keyutils.x86_64 0:1.5.8-3.el7 will be installed
    ---> Package libnfsidmap.x86_64 0:0.25-19.el7 will be installed
    ---> Package libtirpc.x86_64 0:0.2.4-0.16.el7 will be installed
    ---> Package quota.x86_64 1:4.01-19.el7 will be installed
    --> Processing Dependency: quota-nls = 1:4.01-19.el7 for package: 1:quota-4.01-19.el7.x86_64
    --> Processing Dependency: tcp_wrappers for package: 1:quota-4.01-19.el7.x86_64
    ---> Package rpcbind.x86_64 0:0.2.0-48.el7 will be installed
    --> Running transaction check
    ---> Package libbasicobjects.x86_64 0:0.1.1-32.el7 will be installed
    ---> Package libcollection.x86_64 0:0.7.0-32.el7 will be installed
    ---> Package libini_config.x86_64 0:1.3.1-32.el7 will be installed
    --> Processing Dependency: libpath_utils.so.1(PATH_UTILS_0.2.1)(64bit) for package: libini_config-1.3.1-32.el7.x86_64
    --> Processing Dependency: libpath_utils.so.1()(64bit) for package: libini_config-1.3.1-32.el7.x86_64
    ---> Package libref_array.x86_64 0:0.1.5-32.el7 will be installed
    ---> Package libverto-libevent.x86_64 0:0.2.5-4.el7 will be installed
    ---> Package quota-nls.noarch 1:4.01-19.el7 will be installed
    ---> Package tcp_wrappers.x86_64 0:7.6-77.el7 will be installed
    --> Running transaction check
    ---> Package libpath_utils.x86_64 0:0.2.1-32.el7 will be installed
    --> Finished Dependency Resolution
    
    Dependencies Resolved
    
    ==============================================================================================================================================================================================================================================================================
     Package                                                                 Arch                                                         Version                                                                Repository                                                  Size
    ==============================================================================================================================================================================================================================================================================
    Installing:
     nfs-utils                                                               x86_64                                                       1:1.3.0-0.65.el7                                                       base                                                       412 k
    Installing for dependencies:
     gssproxy                                                                x86_64                                                       0.7.0-26.el7                                                           base                                                       110 k
     keyutils                                                                x86_64                                                       1.5.8-3.el7                                                            base                                                        54 k
     libbasicobjects                                                         x86_64                                                       0.1.1-32.el7                                                           base                                                        26 k
     libcollection                                                           x86_64                                                       0.7.0-32.el7                                                           base                                                        42 k
     libini_config                                                           x86_64                                                       1.3.1-32.el7                                                           base                                                        64 k
     libnfsidmap                                                             x86_64                                                       0.25-19.el7                                                            base                                                        50 k
     libpath_utils                                                           x86_64                                                       0.2.1-32.el7                                                           base                                                        28 k
     libref_array                                                            x86_64                                                       0.1.5-32.el7                                                           base                                                        27 k
     libtirpc                                                                x86_64                                                       0.2.4-0.16.el7                                                         base                                                        89 k
     libverto-libevent                                                       x86_64                                                       0.2.5-4.el7                                                            base                                                       8.9 k
     quota                                                                   x86_64                                                       1:4.01-19.el7                                                          base                                                       179 k
     quota-nls                                                               noarch                                                       1:4.01-19.el7                                                          base                                                        90 k
     rpcbind                                                                 x86_64                                                       0.2.0-48.el7                                                           base                                                        60 k
     tcp_wrappers                                                            x86_64                                                       7.6-77.el7                                                             base                                                        78 k
    
    Transaction Summary
    ==============================================================================================================================================================================================================================================================================
    Install  1 Package (+14 Dependent packages)
    
    Total download size: 1.3 M
    Installed size: 3.6 M
    Downloading packages:
    (1/15): libcollection-0.7.0-32.el7.x86_64.rpm                                                                                                                                                                                                          |  42 kB  00:00:00     
    (2/15): gssproxy-0.7.0-26.el7.x86_64.rpm                                                                                                                                                                                                               | 110 kB  00:00:00     
    (3/15): libpath_utils-0.2.1-32.el7.x86_64.rpm                                                                                                                                                                                                          |  28 kB  00:00:00     
    (4/15): libref_array-0.1.5-32.el7.x86_64.rpm                                                                                                                                                                                                           |  27 kB  00:00:00     
    (5/15): keyutils-1.5.8-3.el7.x86_64.rpm                                                                                                                                                                                                                |  54 kB  00:00:00     
    (6/15): libnfsidmap-0.25-19.el7.x86_64.rpm                                                                                                                                                                                                             |  50 kB  00:00:00     
    (7/15): libini_config-1.3.1-32.el7.x86_64.rpm                                                                                                                                                                                                          |  64 kB  00:00:00     
    (8/15): libtirpc-0.2.4-0.16.el7.x86_64.rpm                                                                                                                                                                                                             |  89 kB  00:00:00     
    (9/15): quota-nls-4.01-19.el7.noarch.rpm                                                                                                                                                                                                               |  90 kB  00:00:00     
    (10/15): quota-4.01-19.el7.x86_64.rpm                                                                                                                                                                                                                  | 179 kB  00:00:00     
    (11/15): libbasicobjects-0.1.1-32.el7.x86_64.rpm                                                                                                                                                                                                       |  26 kB  00:00:00     
    (12/15): libverto-libevent-0.2.5-4.el7.x86_64.rpm                                                                                                                                                                                                      | 8.9 kB  00:00:00     
    (13/15): rpcbind-0.2.0-48.el7.x86_64.rpm                                                                                                                                                                                                               |  60 kB  00:00:00     
    (14/15): tcp_wrappers-7.6-77.el7.x86_64.rpm                                                                                                                                                                                                            |  78 kB  00:00:00     
    (15/15): nfs-utils-1.3.0-0.65.el7.x86_64.rpm                                                                                                                                                                                                           | 412 kB  00:00:00     
    ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
    Total                                                                                                                                                                                                                                         2.6 MB/s | 1.3 MB  00:00:00     
    Running transaction check
    Running transaction test
    Transaction test succeeded
    Running transaction
      Installing : libbasicobjects-0.1.1-32.el7.x86_64                                                                                                                                                                                                                       1/15 
      Installing : libref_array-0.1.5-32.el7.x86_64                                                                                                                                                                                                                          2/15 
      Installing : libcollection-0.7.0-32.el7.x86_64                                                                                                                                                                                                                         3/15 
      Installing : libtirpc-0.2.4-0.16.el7.x86_64                                                                                                                                                                                                                            4/15 
      Installing : rpcbind-0.2.0-48.el7.x86_64                                                                                                                                                                                                                               5/15 
      Installing : 1:quota-nls-4.01-19.el7.noarch                                                                                                                                                                                                                            6/15 
      Installing : tcp_wrappers-7.6-77.el7.x86_64                                                                                                                                                                                                                            7/15 
      Installing : 1:quota-4.01-19.el7.x86_64                                                                                                                                                                                                                                8/15 
      Installing : keyutils-1.5.8-3.el7.x86_64                                                                                                                                                                                                                               9/15 
      Installing : libnfsidmap-0.25-19.el7.x86_64                                                                                                                                                                                                                           10/15 
      Installing : libpath_utils-0.2.1-32.el7.x86_64                                                                                                                                                                                                                        11/15 
      Installing : libini_config-1.3.1-32.el7.x86_64                                                                                                                                                                                                                        12/15 
      Installing : libverto-libevent-0.2.5-4.el7.x86_64                                                                                                                                                                                                                     13/15 
      Installing : gssproxy-0.7.0-26.el7.x86_64                                                                                                                                                                                                                             14/15 
      Installing : 1:nfs-utils-1.3.0-0.65.el7.x86_64                                                                                                                                                                                                                        15/15 
      Verifying  : libtirpc-0.2.4-0.16.el7.x86_64                                                                                                                                                                                                                            1/15 
      Verifying  : libverto-libevent-0.2.5-4.el7.x86_64                                                                                                                                                                                                                      2/15 
      Verifying  : 1:quota-4.01-19.el7.x86_64                                                                                                                                                                                                                                3/15 
      Verifying  : gssproxy-0.7.0-26.el7.x86_64                                                                                                                                                                                                                              4/15 
      Verifying  : libpath_utils-0.2.1-32.el7.x86_64                                                                                                                                                                                                                         5/15 
      Verifying  : libnfsidmap-0.25-19.el7.x86_64                                                                                                                                                                                                                            6/15 
      Verifying  : keyutils-1.5.8-3.el7.x86_64                                                                                                                                                                                                                               7/15 
      Verifying  : 1:nfs-utils-1.3.0-0.65.el7.x86_64                                                                                                                                                                                                                         8/15 
      Verifying  : tcp_wrappers-7.6-77.el7.x86_64                                                                                                                                                                                                                            9/15 
      Verifying  : libcollection-0.7.0-32.el7.x86_64                                                                                                                                                                                                                        10/15 
      Verifying  : libref_array-0.1.5-32.el7.x86_64                                                                                                                                                                                                                         11/15 
      Verifying  : libbasicobjects-0.1.1-32.el7.x86_64                                                                                                                                                                                                                      12/15 
      Verifying  : rpcbind-0.2.0-48.el7.x86_64                                                                                                                                                                                                                              13/15 
      Verifying  : libini_config-1.3.1-32.el7.x86_64                                                                                                                                                                                                                        14/15 
      Verifying  : 1:quota-nls-4.01-19.el7.noarch                                                                                                                                                                                                                           15/15 
    
    Installed:
      nfs-utils.x86_64 1:1.3.0-0.65.el7                                                                                                                                                                                                                                           
    
    Dependency Installed:
      gssproxy.x86_64 0:0.7.0-26.el7        keyutils.x86_64 0:1.5.8-3.el7       libbasicobjects.x86_64 0:0.1.1-32.el7     libcollection.x86_64 0:0.7.0-32.el7    libini_config.x86_64 0:1.3.1-32.el7    libnfsidmap.x86_64 0:0.25-19.el7    libpath_utils.x86_64 0:0.2.1-32.el7   
      libref_array.x86_64 0:0.1.5-32.el7    libtirpc.x86_64 0:0.2.4-0.16.el7    libverto-libevent.x86_64 0:0.2.5-4.el7    quota.x86_64 1:4.01-19.el7             quota-nls.noarch 1:4.01-19.el7         rpcbind.x86_64 0:0.2.0-48.el7       tcp_wrappers.x86_64 0:7.6-77.el7      
    
    Complete!
    [root@node203.yinzhengjie.org.cn ~]# 
    [root@node203.yinzhengjie.org.cn ~]# yum -y install nfs-utils
    [root@master200.yinzhengjie.org.cn ~]# mount -t nfs 172.200.1.110:/yinzhengjie/data/volume1 /mnt      #在K8S node节点测试NFS服务是否可以成功挂载
    [root@master200.yinzhengjie.org.cn ~]# 
    [root@master200.yinzhengjie.org.cn ~]# mount | grep mnt
    172.200.1.110:/yinzhengjie/data/volume1 on /mnt type nfs4 (rw,relatime,vers=4.1,rsize=524288,wsize=524288,namlen=255,hard,proto=tcp,timeo=600,retrans=2,sec=sys,clientaddr=172.200.1.200,local_lock=none,addr=172.200.1.110)
    [root@master200.yinzhengjie.org.cn ~]# 
    [root@master200.yinzhengjie.org.cn ~]# df -h | grep mnt
    172.200.1.110:/yinzhengjie/data/volume1  1.6T  416M  1.6T   1% /mnt
    [root@master200.yinzhengjie.org.cn ~]# 
    [root@master200.yinzhengjie.org.cn ~]# 
    [root@master200.yinzhengjie.org.cn ~]# mount -t nfs 172.200.1.110:/yinzhengjie/data/volume1 /mnt      #在K8S node节点测试NFS服务是否可以成功挂载,即验证可用性。

    3>.基于NFS存储卷创建Pod

    [root@master200.yinzhengjie.org.cn ~]# kubectl explain pods.spec.volumes.nfs
    KIND:     Pod
    VERSION:  v1
    
    RESOURCE: nfs <Object>
    
    DESCRIPTION:
         NFS represents an NFS mount on the host that shares a pod's lifetime More
         info: https://kubernetes.io/docs/concepts/storage/volumes#nfs
    
         Represents an NFS mount that lasts the lifetime of a pod. NFS volumes do
         not support ownership management or SELinux relabeling.
    
    FIELDS:
       path    <string> -required-
         Path that is exported by the NFS server. More info:
         https://kubernetes.io/docs/concepts/storage/volumes#nfs
    
       readOnly    <boolean>
         ReadOnly here will force the NFS export to be mounted with read-only
         permissions. Defaults to false. More info:
         https://kubernetes.io/docs/concepts/storage/volumes#nfs
    
       server    <string> -required-
         Server is the hostname or IP address of the NFS server. More info:
         https://kubernetes.io/docs/concepts/storage/volumes#nfs
    
    [root@master200.yinzhengjie.org.cn ~]# 
    [root@master200.yinzhengjie.org.cn ~]# kubectl explain pods.spec.volumes.nfs
    [root@master200.yinzhengjie.org.cn ~]# vim /yinzhengjie/data/k8s/manifests/basic/volumes/pod-volume-demo3.yaml
    [root@master200.yinzhengjie.org.cn ~]# 
    [root@master200.yinzhengjie.org.cn ~]# cat /yinzhengjie/data/k8s/manifests/basic/volumes/pod-volume-demo3.yaml
    apiVersion: v1
    kind: Pod
    metadata:
      name: vol-nfs-pod
      namespace: yinzhengjie-volume
      labels:
        app: redis
    spec:
      containers:
      - name: redis
        image: redis:alpine
        ports:
        - containerPort: 6379
          name: redis
        volumeMounts:
        - mountPath: /data
          name: redisdata
      volumes:
        - name: redisdata
          nfs:
            server: 172.200.1.110
            path: /yinzhengjie/data/volume1
    [root@master200.yinzhengjie.org.cn ~]# 
    [root@master200.yinzhengjie.org.cn ~]# vim /yinzhengjie/data/k8s/manifests/basic/volumes/pod-volume-demo3.yaml
    [root@master200.yinzhengjie.org.cn ~]# kubectl apply -f /yinzhengjie/data/k8s/manifests/basic/volumes/pod-volume-demo3.yaml
    pod/vol-nfs-pod created
    [root@master200.yinzhengjie.org.cn ~]# 
    [root@master200.yinzhengjie.org.cn ~]# kubectl get pods -n yinzhengjie-volume
    NAME               READY   STATUS    RESTARTS   AGE
    myapp              1/1     Running   0          177m
    vol-emptydir-pod   2/2     Running   0          71m
    vol-nfs-pod        1/1     Running   0          16s
    [root@master200.yinzhengjie.org.cn ~]# 
    [root@master200.yinzhengjie.org.cn ~]# kubectl apply -f /yinzhengjie/data/k8s/manifests/basic/volumes/pod-volume-demo3.yaml
    [root@master200.yinzhengjie.org.cn ~]# kubectl get pods -n yinzhengjie-volume -o wide
    NAME               READY   STATUS    RESTARTS   AGE    IP            NODE                         NOMINATED NODE   READINESS GATES
    myapp              1/1     Running   0          177m   10.244.3.13   node203.yinzhengjie.org.cn   <none>           <none>
    vol-emptydir-pod   2/2     Running   0          71m    10.244.1.22   node201.yinzhengjie.org.cn   <none>           <none>
    vol-nfs-pod        1/1     Running   0          59s    10.244.2.19   node202.yinzhengjie.org.cn   <none>           <none>
    [root@master200.yinzhengjie.org.cn ~]# 
    [root@master200.yinzhengjie.org.cn ~]# 
    [root@master200.yinzhengjie.org.cn ~]# kubectl get pods -n yinzhengjie-volume -o wide
    [root@master200.yinzhengjie.org.cn ~]# kubectl describe pods vol-nfs-pod -n yinzhengjie-volume 
    Name:         vol-nfs-pod
    Namespace:    yinzhengjie-volume
    Priority:     0
    Node:         node202.yinzhengjie.org.cn/172.200.1.202
    Start Time:   Sun, 09 Feb 2020 10:22:09 +0800
    Labels:       app=redis
    Annotations:  kubectl.kubernetes.io/last-applied-configuration:
                    {"apiVersion":"v1","kind":"Pod","metadata":{"annotations":{},"labels":{"app":"redis"},"name":"vol-nfs-pod","namespace":"yinzhengjie-volume...
    Status:       Running
    IP:           10.244.2.19
    IPs:
      IP:  10.244.2.19
    Containers:
      redis:
        Container ID:   docker://1bd821c9cdada24d89d66f9650e2a6957181e121f30de3bd68f217ed4eaa1de5
        Image:          redis:alpine
        Image ID:       docker-pullable://redis@sha256:cb9783b1c39bb34f8d6572406139ab325c4fac0b28aaa25d5350495637bb2f76
        Port:           6379/TCP
        Host Port:      0/TCP
        State:          Running
          Started:      Sun, 09 Feb 2020 10:22:10 +0800
        Ready:          True
        Restart Count:  0
        Environment:    <none>
        Mounts:
          /data from redisdata (rw)
          /var/run/secrets/kubernetes.io/serviceaccount from default-token-bxflv (ro)
    Conditions:
      Type              Status
      Initialized       True 
      Ready             True 
      ContainersReady   True 
      PodScheduled      True 
    Volumes:
      redisdata:
        Type:      NFS (an NFS mount that lasts the lifetime of a pod)
        Server:    172.200.1.110
        Path:      /yinzhengjie/data/volume1
        ReadOnly:  false
      default-token-bxflv:
        Type:        Secret (a volume populated by a Secret)
        SecretName:  default-token-bxflv
        Optional:    false
    QoS Class:       BestEffort
    Node-Selectors:  <none>
    Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                     node.kubernetes.io/unreachable:NoExecute for 300s
    Events:
      Type    Reason     Age        From                                 Message
      ----    ------     ----       ----                                 -------
      Normal  Scheduled  <unknown>  default-scheduler                    Successfully assigned yinzhengjie-volume/vol-nfs-pod to node202.yinzhengjie.org.cn
      Normal  Pulled     2m2s       kubelet, node202.yinzhengjie.org.cn  Container image "redis:alpine" already present on machine
      Normal  Created    2m2s       kubelet, node202.yinzhengjie.org.cn  Created container redis
      Normal  Started    2m2s       kubelet, node202.yinzhengjie.org.cn  Started container redis
    [root@master200.yinzhengjie.org.cn ~]# 
    [root@master200.yinzhengjie.org.cn ~]# kubectl describe pods vol-nfs-pod -n yinzhengjie-volume

    4>.重新创建Pod并验证Redis的测试数据

    [root@master200.yinzhengjie.org.cn ~]# kubectl get pods -n yinzhengjie-volume -o wide
    NAME               READY   STATUS    RESTARTS   AGE     IP            NODE                         NOMINATED NODE   READINESS GATES
    myapp              1/1     Running   0          3h6m    10.244.3.13   node203.yinzhengjie.org.cn   <none>           <none>
    vol-emptydir-pod   2/2     Running   0          79m     10.244.1.22   node201.yinzhengjie.org.cn   <none>           <none>
    vol-nfs-pod        1/1     Running   0          9m11s   10.244.2.19   node202.yinzhengjie.org.cn   <none>           <none>
    [root@master200.yinzhengjie.org.cn ~]# 
    [root@master200.yinzhengjie.org.cn ~]# kubectl exec -it vol-nfs-pod -n yinzhengjie-volume -- /bin/sh
    /data # 
    /data # 
    /data # redis-cli 
    127.0.0.1:6379> 
    127.0.0.1:6379> SET myblog "https://www.cnblogs.com/yinzhengjie/"
    OK
    127.0.0.1:6379> 
    127.0.0.1:6379> BGSAVE
    Background saving started
    127.0.0.1:6379> 
    127.0.0.1:6379> exit
    /data # 
    /data # ls
    dump.rdb
    /data # 
    /data # exit
    [root@node203.yinzhengjie.org.cn ~]# 
    [root@node203.yinzhengjie.org.cn ~]# 
    [root@master200.yinzhengjie.org.cn ~]# kubectl exec -it vol-nfs-pod -n yinzhengjie-volume -- /bin/sh                    #登录Reids服务的pod并创建测试数据
    [root@master200.yinzhengjie.org.cn ~]# kubectl get pods -n yinzhengjie-volume -o wide
    NAME               READY   STATUS    RESTARTS   AGE     IP            NODE                         NOMINATED NODE   READINESS GATES
    myapp              1/1     Running   0          3h6m    10.244.3.13   node203.yinzhengjie.org.cn   <none>           <none>
    vol-emptydir-pod   2/2     Running   0          79m     10.244.1.22   node201.yinzhengjie.org.cn   <none>           <none>
    vol-nfs-pod        1/1     Running   0          9m11s   10.244.2.19   node202.yinzhengjie.org.cn   <none>           <none>
    [root@master200.yinzhengjie.org.cn ~]# 
    [root@master200.yinzhengjie.org.cn ~]# kubectl delete -f /yinzhengjie/data/k8s/manifests/basic/volumes/pod-volume-demo3.yaml
    pod "vol-nfs-pod" deleted
    [root@master200.yinzhengjie.org.cn ~]# 
    [root@master200.yinzhengjie.org.cn ~]# kubectl get pods -n yinzhengjie-volume -o wide
    NAME               READY   STATUS    RESTARTS   AGE    IP            NODE                         NOMINATED NODE   READINESS GATES
    myapp              1/1     Running   0          3h9m   10.244.3.13   node203.yinzhengjie.org.cn   <none>           <none>
    vol-emptydir-pod   2/2     Running   0          83m    10.244.1.22   node201.yinzhengjie.org.cn   <none>           <none>
    [root@master200.yinzhengjie.org.cn ~]# 
    [root@master200.yinzhengjie.org.cn ~]# 
    [root@master200.yinzhengjie.org.cn ~]# kubectl delete -f /yinzhengjie/data/k8s/manifests/basic/volumes/pod-volume-demo3.yaml      #删除redis的Pod
    [root@master200.yinzhengjie.org.cn ~]# 
    [root@master200.yinzhengjie.org.cn ~]# vim /yinzhengjie/data/k8s/manifests/basic/volumes/pod-volume-demo3.yaml
    [root@master200.yinzhengjie.org.cn ~]# 
    [root@master200.yinzhengjie.org.cn ~]# cat /yinzhengjie/data/k8s/manifests/basic/volumes/pod-volume-demo3.yaml
    apiVersion: v1
    kind: Pod
    metadata:
      name: vol-nfs-pod
      namespace: yinzhengjie-volume
      labels:
        app: redis
    spec:
      nodeName: node201.yinzhengjie.org.cn
      containers:
      - name: redis
        image: redis:alpine
        ports:
        - containerPort: 6379
          name: redis
        volumeMounts:
        - mountPath: /data
          name: redisdata
      volumes:
        - name: redisdata
          nfs:
            server: 172.200.1.110
            path: /yinzhengjie/data/volume1
    [root@master200.yinzhengjie.org.cn ~]# 
    [root@master200.yinzhengjie.org.cn ~]# vim /yinzhengjie/data/k8s/manifests/basic/volumes/pod-volume-demo3.yaml              #再次编辑配置文件指定nodeName参数

    5>.查看NFS中的数据

    五.其它存储案例

      博主推荐案例:
        https://www.cnblogs.com/yinzhengjie/p/12286111.html
  • 相关阅读:
    Eclipse配置Maven3.5
    VM搭建Hadoop环境静态IP未起作用
    Word2010制作倒福字
    Word2010制作日历
    【整理】HTML5游戏开发学习笔记(1)- 骰子游戏
    [转载]HTML5开发入门经典教程和案例合集(含视频教程)
    [转载]如何做到 jQuery-free?
    [转载]教你如何塑造JavaScript牛逼形象
    Cloud9免费云IDE代码编辑平台空间支持Node.js,PHP,Python可使用FTP管理
    [转载]PayPal为什么从Java迁移到Node.js,性能提高一倍,文件代码减少44%
  • 原文地址:https://www.cnblogs.com/yinzhengjie/p/12275463.html
Copyright © 2011-2022 走看看