zoukankan html css js c++ java

Kubernetes入门与实战：01

1、Kubernetes基本概览

　　由于Kubernetes每年三四个版本发布，官网永远是最权威的；话不多说，先提供官网地址：https://kubernetes.io/zh/docs/tutorials/kubernetes-basics/

1.1、集群与节点

查看master主键角色：

1 [root@master1 ~]# kubectl get componentstatuses     　　　　#  也可以使用简写： kubectl get cs
2 Warning: v1 ComponentStatus is deprecated in v1.19+
3 NAME                 STATUS      MESSAGE                                                                                       ERROR
4 scheduler            Unhealthy   Get "http://127.0.0.1:10251/healthz": dial tcp 127.0.0.1:10251: connect: connection refused   
5 controller-manager   Healthy     ok                                                                                            
6 etcd-0               Healthy     {"health":"true","reason":""}                                                                 
7 [root@master1 ~]#

查看node节点列表：

1 [root@master1 ~]# kubectl get nodes -o wide　　　　　　　　　　#  也可以使用简写： kubectl get no
2 NAME      STATUS   ROLES                  AGE   VERSION   INTERNAL-IP    EXTERNAL-IP   OS-IMAGE         KERNEL-VERSION          CONTAINER-RUNTIME
3 master1   Ready    control-plane,master   44h   v1.22.2   192.168.1.66   <none>        CentOS Linux 8   4.18.0-240.el8.x86_64   docker://20.10.8
4 node1     Ready    <none>                 44h   v1.22.2   192.168.1.77   <none>        CentOS Linux 8   4.18.0-240.el8.x86_64   docker://20.10.8
5 node2     Ready    <none>                 44h   v1.22.2   192.168.1.88   <none>        CentOS Linux 8   4.18.0-240.el8.x86_64   docker://20.10.8
6 [root@master1 ~]#

查看node节点详情：

 1 [root@master1 ~]# kubectl describe nodes node2
 2 Name:               node2
 3 Roles:              <none>
 4 Labels:             beta.kubernetes.io/arch=amd64
 5                     beta.kubernetes.io/os=linux
 6                     kubernetes.io/arch=amd64
 7                     kubernetes.io/hostname=node2
 8                     kubernetes.io/os=linux
 9 Annotations:        flannel.alpha.coreos.com/backend-data: {"VNI":1,"VtepMAC":"be:68:4a:f9:ac:de"}
10                     flannel.alpha.coreos.com/backend-type: vxlan
11                     flannel.alpha.coreos.com/kube-subnet-manager: true
12                     flannel.alpha.coreos.com/public-ip: 192.168.1.88
13                     kubeadm.alpha.kubernetes.io/cri-socket: /var/run/dockershim.sock
14                     node.alpha.kubernetes.io/ttl: 0
15                     volumes.kubernetes.io/controller-managed-attach-detach: true
16 CreationTimestamp:  Fri, 01 Oct 2021 06:59:33 -0400
17 Taints:             <none>
18 Unschedulable:      false
19 Lease:
20   HolderIdentity:  node2
21   AcquireTime:     <unset>
22   RenewTime:       Sun, 03 Oct 2021 03:45:59 -0400
23 Conditions:
24   Type                 Status  LastHeartbeatTime                 LastTransitionTime                Reason                       Message
25   ----                 ------  -----------------                 ------------------                ------                       -------
26   NetworkUnavailable   False   Sun, 03 Oct 2021 02:47:43 -0400   Sun, 03 Oct 2021 02:47:43 -0400   FlannelIsUp                  Flannel is running on this node
27   MemoryPressure       False   Sun, 03 Oct 2021 03:42:49 -0400   Fri, 01 Oct 2021 06:59:33 -0400   KubeletHasSufficientMemory   kubelet has sufficient memory available
28   DiskPressure         False   Sun, 03 Oct 2021 03:42:49 -0400   Fri, 01 Oct 2021 06:59:33 -0400   KubeletHasNoDiskPressure     kubelet has no disk pressure
29   PIDPressure          False   Sun, 03 Oct 2021 03:42:49 -0400   Fri, 01 Oct 2021 06:59:33 -0400   KubeletHasSufficientPID      kubelet has sufficient PID available
30   Ready                True    Sun, 03 Oct 2021 03:42:49 -0400   Fri, 01 Oct 2021 07:03:54 -0400   KubeletReady                 kubelet is posting ready status
31 Addresses:　　　　　　　　# 节点的IP地址和主机名
32   InternalIP:  192.168.1.88
33   Hostname:    node2
34 Capacity:　　　　　　　　# 节点总资源
35   cpu:                2
36   ephemeral-storage:  17398Mi
37   hugepages-1Gi:      0
38   hugepages-2Mi:      0
39   memory:             1996236Ki
40   pods:               110
41 Allocatable:　　　　　　# 节点可以分配的资源
42   cpu:                2
43   ephemeral-storage:  16418812697
44   hugepages-1Gi:      0
45   hugepages-2Mi:      0
46   memory:             1893836Ki
47   pods:               110
48 System Info:　　　　　　# 系统的相关信息
49   Machine ID:                 fd49b99f63534bc9a6544ff2f1d68ef5
50   System UUID:                2a454d56-033c-c61e-c4f8-37b171f12218
51   Boot ID:                    c66bb70c-b358-4d60-b6f2-9f2cb49dad2e
52   Kernel Version:             4.18.0-240.el8.x86_64
53   OS Image:                   CentOS Linux 8
54   Operating System:           linux
55   Architecture:               amd64
56   Container Runtime Version:  docker://20.10.8
57   Kubelet Version:            v1.22.2　　　　　　　　　　# kubelet版本
58   Kube-Proxy Version:         v1.22.2　　　　　　　　　　# kube-proxy版本
59 PodCIDR:                      172.16.2.0/24　　　　　　# pod网络
60 PodCIDRs:                     172.16.2.0/24
61 Non-terminated Pods:          (2 in total)　　　　　　 # 当前节点已经运行pod情况，及每个pod占用的资源情况　
62   Namespace                   Name                     CPU Requests  CPU Limits  Memory Requests  Memory Limits  Age
63   ---------                   ----                     ------------  ----------  ---------------  -------------  ---
64   kube-system                 kube-flannel-ds-zp785    100m (5%)     100m (5%)   50Mi (2%)        50Mi (2%)      44h
65   kube-system                 kube-proxy-s6ndp         0 (0%)        0 (0%)      0 (0%)           0 (0%)         53m
66 Allocated resources:　　　　# 已分配资源情况
67   (Total limits may be over 100 percent, i.e., overcommitted.)
68   Resource           Requests   Limits
69   --------           --------   ------
70   cpu                100m (5%)  100m (5%)
71   memory             50Mi (2%)  50Mi (2%)
72   ephemeral-storage  0 (0%)     0 (0%)
73   hugepages-1Gi      0 (0%)     0 (0%)
74   hugepages-2Mi      0 (0%)     0 (0%)
75 Events:
76   Type    Reason                   Age                From     Message
77   ----    ------                   ----               ----     -------
78   Normal  Starting                 44h                kubelet  Starting kubelet.
79   Normal  NodeHasSufficientMemory  44h (x2 over 44h)  kubelet  Node node2 status is now: NodeHasSufficientMemory
80   Normal  NodeHasNoDiskPressure    44h (x2 over 44h)  kubelet  Node node2 status is now: NodeHasNoDiskPressure
81   Normal  NodeHasSufficientPID     44h (x2 over 44h)  kubelet  Node node2 status is now: NodeHasSufficientPID
82   Normal  NodeAllocatableEnforced  44h                kubelet  Updated Node Allocatable limit across pods
83   Normal  NodeReady                44h                kubelet  Node node2 status is now: NodeReady
84 [root@master1 ~]#

1.2、容器与应用

　　kubernetes是容器编排引擎，其负责容器的调度，管理和容器的运行，但kubernetes调度最小单位并非是container，而是pod，pod中可包含多个container，通常集群中不会直接运行pod，而是通过各种工作负载的控制器如Deployments，ReplicaSets，DaemonSets的方式运行，为啥？因为控制器能够保证pod状态的一致性，正如官方所描述的一样“make sure the current state match to the desire state”，确保当前状态和预期的一致，简单来说就是pod异常了，控制器会在其他节点重建，确保集群当前运行的pod和预期设定的一致。

pod是kubernetes中运行的最小单元
pod中包含一个容器或者多个容器
pod不会单独使用，需要有工作负载来控制，如Deployments，StatefulSets，DaemonSets，CronJobs等

Pod概览：

1.3、服务访问

　　kubernetes中pod是实际运行的载体，pod依附于node中，node可能会出现故障，kubernetes的控制器如replicasets会在其他node上重新拉起一个pod，新的pod会分配一个新的IP；再者，应用部署时会包含多个副本replicas，如同个应用deployments部署了3个pod副本，pod相当于后端的Real Server，如何实现这三个应用访问呢？对于这种情况，我们一般会在Real Server前面加一个负载均衡Load Balancer，service就是pod的负载均衡调度器，service将动态的pod抽象为一个服务，应用程序直接访问service即可，service会自动将请求转发到后端的pod。负责service转发规则有两种机制：iptables和ipvs，iptables通过设置DNAT等规则实现负载均衡，ipvs通过ipvsadm设置转发规。

Service 和 Label：

根据服务不同的访问方式，service分为如下几种类型：ClusterIP，NodePort，LoadBalancer和_ExternalName，可通过type设置：

ClusterIP，集群内部互访，与DNS结合实现集群内部的服务发现；
NodePort，通过NAT将每个node节点暴露一个端口实现外部访问；
LoadBalancer，实现云厂商外部接入方式的接口，需要依赖云服务提供商实现具体技术细节，如腾讯云实现与CLB集成；
ExternalName，通过服务名字暴露服务名，当前可由ingress实现，将外部的请求以域名转发的形式转发到集群，需要依附具体的外部实现，如nginx，traefik,各大云计算厂商实现接入细节；

　　pod是动态变化的，ip地址可能会变化（如node故障），副本数可能会变化，如应用扩展scale up，应用锁容scale down等，service如何识别到pod的动态变化呢？答案是labels，通过labels自动会过滤出某个应用的Endpoints，当pod变化时会自动更新Endpoints，不同的应用会有由不同的label组成。labels相关可以参考下 https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/

2、创建应用

　　下面开始部署一个应用即deployments，kubernetes中包含各种workload如无状态话的Deployments，有状态化的StatefulSets，守护进程的DaemonSets，每种workload对应不同的应用场景，我们先以Deployments为例入门，其他workload均以此类似，一般而言，在kubernetes中部署应用均以yaml文件方式部署，对于初学者而言，编写yaml文件太冗长，不适合初学，我们先kubectl命令行方式实现API的接入。

2.1、创建两个副本的nginx的应用并查看是否处于：AVAILABLE状态：

1 [root@master1 ~]# kubectl create deployment demo-nginx --image=nginx:1.8.1 --replicas=2
2 deployment.apps/demo-nginx created
3 [root@master1 ~]# 
4 [root@master1 ~]# kubectl get deployments.apps 
5 NAME         READY   UP-TO-DATE   AVAILABLE   AGE
6 demo-nginx   2/2     2            2           59s
7 [root@master1 ~]#

2.2、查看应用的详细信息,我们可以知道Deployments是通过ReplicaSets控制副本数的，由Replicaset控制pod数：

 1 [root@master1 ~]# kubectl describe deployments.apps demo-nginx 
 2 Name:                   demo-nginx
 3 Namespace:              default
 4 CreationTimestamp:      Sun, 03 Oct 2021 04:13:57 -0400
 5 Labels:                 app=demo-nginx　　# 标签，后面的service选择的依据
 6 Annotations:            deployment.kubernetes.io/revision: 1
 7 Selector:               app=demo-nginx
 8 Replicas:               2 desired | 2 updated | 2 total | 2 available | 0 unavailable
 9 StrategyType:           RollingUpdate　　# 滚动升级的策略
10 MinReadySeconds:        0
11 RollingUpdateStrategy:  25% max unavailable, 25% max surge
12 Pod Template:　　　　　# 容器应用模版，包含镜像，port，存储等　　　　　
13   Labels:  app=demo-nginx
14   Containers:
15    nginx:
16     Image:        nginx:1.8.1
17     Port:         <none>
18     Host Port:    <none>
19     Environment:  <none>
20     Mounts:       <none>
21   Volumes:        <none>
22 Conditions:
23   Type           Status  Reason
24   ----           ------  ------
25   Available      True    MinimumReplicasAvailable
26   Progressing    True    NewReplicaSetAvailable
27 OldReplicaSets:  <none>
28 NewReplicaSet:   demo-nginx-55ccc46db6 (2/2 replicas created)　　# 控制器的名称
29 Events:
30   Type    Reason             Age   From                   Message
31   ----    ------             ----  ----                   -------
32   Normal  ScalingReplicaSet  78s   deployment-controller  Scaled up replica set demo-nginx-55ccc46db6 to 2
33 [root@master1 ~]#

2.3、查看replicasets情况，通过查看可知replicasets副本控制器生成了两个pod：

 1 [root@master1 ~]# kubectl get replicasets.apps 
 2 NAME                    DESIRED   CURRENT   READY   AGE
 3 demo-nginx-55ccc46db6   2         2         2       112s
 4 [root@master1 ~]# 
 5 [root@master1 ~]# kubectl describe replicasets.apps demo-nginx-55ccc46db6 
 6 Name:           demo-nginx-55ccc46db6
 7 Namespace:      default
 8 Selector:       app=demo-nginx,pod-template-hash=55ccc46db6
 9 Labels:         app=demo-nginx
10                 pod-template-hash=55ccc46db6
11 Annotations:    deployment.kubernetes.io/desired-replicas: 2　　# 滚动升级的信息，副本树，最大数，应用版本
12                 deployment.kubernetes.io/max-replicas: 3
13                 deployment.kubernetes.io/revision: 1
14 Controlled By:  Deployment/demo-nginx　　　　# 控制器类型
15 Replicas:       2 current / 2 desired　　　　# 当前和期望的pod数量
16 Pods Status:    2 Running / 0 Waiting / 0 Succeeded / 0 Failed
17 Pod Template:
18   Labels:  app=demo-nginx
19            pod-template-hash=55ccc46db6
20   Containers:
21    nginx:
22     Image:        nginx:1.8.1
23     Port:         <none>
24     Host Port:    <none>
25     Environment:  <none>
26     Mounts:       <none>
27   Volumes:        <none>
28 Events:　　　　　　# 日志事件：生成了三个不同的pod
29   Type    Reason            Age   From                   Message
30   ----    ------            ----  ----                   -------
31   Normal  SuccessfulCreate  2m7s  replicaset-controller  Created pod: demo-nginx-55ccc46db6-lg4nz
32   Normal  SuccessfulCreate  2m7s  replicaset-controller  Created pod: demo-nginx-55ccc46db6-jmjkx
33 [root@master1 ~]#

2.4、查看pod的情况，实际应用部署的载体，pod中部署了一个nginx的容器并分配了一个ip，可通过该ip直接访问应用：

 1 [root@master1 ~]# kubectl get pods -o wide   　　　　# 每个pod的名称前缀和上面的replicasets一样
 2 NAME                          READY   STATUS    RESTARTS   AGE   IP           NODE    NOMINATED NODE   READINESS GATES
 3 demo-nginx-55ccc46db6-jmjkx   1/1     Running   0          20m   172.16.2.4   node2   <none>           <none>
 4 demo-nginx-55ccc46db6-lg4nz   1/1     Running   0          20m   172.16.1.5   node1   <none>           <none>
 5 [root@master1 ~]# 
 6 [root@master1 ~]# kubectl describe pods demo-nginx-55ccc46db6-jmjkx 
 7 Name:         demo-nginx-55ccc46db6-jmjkx
 8 Namespace:    default
 9 Priority:     0
10 Node:         node2/192.168.1.88
11 Start Time:   Sun, 03 Oct 2021 04:13:57 -0400
12 Labels:       app=demo-nginx
13               pod-template-hash=55ccc46db6
14 Annotations:  <none>
15 Status:       Running
16 IP:           172.16.2.4　　# pod的IP地址
17 IPs:
18   IP:           172.16.2.4
19 Controlled By:  ReplicaSet/demo-nginx-55ccc46db6
20 Containers:
21   nginx:
22     Container ID:   docker://430d6e35fb2bb8e4cce6d498591da05748592ad254182c19dc87e41fca951dfd
23     Image:          nginx:1.8.1
24     Image ID:       docker-pullable://nginx@sha256:9b3e9f189890ef9d6713c3384da3809731bdb0bff84e7b68da330ebadf533085
25     Port:           <none>
26     Host Port:      <none>
27     State:          Running
28       Started:      Sun, 03 Oct 2021 04:14:28 -0400
29     Ready:          True
30     Restart Count:  0
31     Environment:    <none>
32     Mounts:
33       /var/run/secrets/kubernetes.io/serviceaccount from kube-api-access-n2r59 (ro)
34 Conditions:
35   Type              Status
36   Initialized       True 
37   Ready             True 
38   ContainersReady   True 
39   PodScheduled      True 
40 Volumes:
41   kube-api-access-n2r59:
42     Type:                    Projected (a volume that contains injected data from multiple sources)
43     TokenExpirationSeconds:  3607
44     ConfigMapName:           kube-root-ca.crt
45     ConfigMapOptional:       <nil>
46     DownwardAPI:             true
47 QoS Class:                   BestEffort
48 Node-Selectors:              <none>
49 Tolerations:                 node.kubernetes.io/not-ready:NoExecute op=Exists for 300s
50                              node.kubernetes.io/unreachable:NoExecute op=Exists for 300s
51 Events:
52   Type    Reason     Age   From               Message
53   ----    ------     ----  ----               -------
54   Normal  Scheduled  21m   default-scheduler  Successfully assigned default/demo-nginx-55ccc46db6-jmjkx to node2
55   Normal  Pulling    21m   kubelet            Pulling image "nginx:1.8.1"
56   Normal  Pulled     21m   kubelet            Successfully pulled image "nginx:1.8.1" in 30.161859239s
57   Normal  Created    21m   kubelet            Created container nginx
58   Normal  Started    21m   kubelet            Started container nginx
59 [root@master1 ~]# 
60 [root@master1 ~]# curl 172.16.2.4　　# 通过pod的IP地址直接访问
61 <!DOCTYPE html>
62 <html>
63 <head>
64 <title>Welcome to nginx!</title>
65 <style>
66     body {
67          35em;
68         margin: 0 auto;
69         font-family: Tahoma, Verdana, Arial, sans-serif;
70     }
71 </style>
72 </head>
73 <body>
74 <h1>Welcome to nginx!</h1>
75 <p>If you see this page, the nginx web server is successfully installed and
76 working. Further configuration is required.</p>
77 
78 <p>For online documentation and support please refer to
79 <a href="http://nginx.org/">nginx.org</a>.<br/>
80 Commercial support is available at
81 <a href="http://nginx.com/">nginx.com</a>.</p>
82 
83 <p><em>Thank you for using nginx.</em></p>
84 </body>
85 </html>
86 [root@master1 ~]#

3、访问应用

　　kubernetes为每个pod都分配了一个ip地址，可通过该地址直接访问应用，相当于访问RS，但一个应用是一个整体，由多个副本数组成，需要依赖于service来实现应用的负载均衡，service我们探讨ClusterIP和NodePort的访问方式。

3.1、设置pod的内容，为了方便区分，我们将两个pod的nginx站点内容设置为不同，以观察通过各自的IP的访问效果：

 1 [root@master1 ~]# kubectl get pods -o wide
 2 NAME                          READY   STATUS    RESTARTS   AGE   IP           NODE    NOMINATED NODE   READINESS GATES
 3 demo-nginx-55ccc46db6-jmjkx   1/1     Running   0          35m   172.16.2.4   node2   <none>           <none>
 4 demo-nginx-55ccc46db6-lg4nz   1/1     Running   0          35m   172.16.1.5   node1   <none>           <none>
 5 [root@master1 ~]# 
 6 [root@master1 ~]# kubectl exec -it demo-nginx-55ccc46db6-jmjkx /bin/bash　　　　# 修改网页内容为：nginx-web1
 7 kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
 8 root@demo-nginx-55ccc46db6-jmjkx:/# echo 'nginx-web1' > /usr/share/nginx/html/index.html 
 9 root@demo-nginx-55ccc46db6-jmjkx:/# exit
10 exit
11 [root@master1 ~]# kubectl exec -it demo-nginx-55ccc46db6-lg4nz /bin/bash　　　　# 修改网页内容为：nginx-web2
12 kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
13 root@demo-nginx-55ccc46db6-lg4nz:/# echo 'nginx-web2' > /usr/share/nginx/html/index.html
14 root@demo-nginx-55ccc46db6-lg4nz:/# exit
15 exit
16 [root@master1 ~]# # 验证访问结果
17 [root@master1 ~]# curl http://172.16.2.4
18 nginx-web1
19 [root@master1 ~]# curl http://172.16.1.5
20 nginx-web2
21 [root@master1 ~]#

3.2、通过ClusterIP访问

　　通过pod的ip直接访问应用，对于单个pod的应用可以实现，对于多个副本replicas的应用则不符合要求，需要通过service来实现负载均衡，service需要设置不同的type，默认为ClusterIP即集群内部访问，如下通过expose子命令将服务暴露到service：

暴露service,其中port表示代理监听端口，target-port代表是容器的端口，type设置的是service的类型，查看service的详情，可以看到service通过labels选择器selector自动将pod的ip生成endpoints

 1 [root@master1 ~]# kubectl expose deployment demo-nginx --port=80 --protocol=TCP --type ClusterIP
 2 service/demo-nginx exposed
 3 [root@master1 ~]# 
 4 [root@master1 ~]# kubectl get svc        # 查看service列表，显示有两个，kubernetes为默认集群创建的service
 5 NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
 6 demo-nginx   ClusterIP   10.96.199.219   <none>        80/TCP    15s
 7 kubernetes   ClusterIP   10.96.0.1       <none>        443/TCP   46h
 8 [root@master1 ~]# 
 9 [root@master1 ~]# kubectl describe service demo-nginx     # 查看service详情，可以看到Labels的Seletor和前面Deployments设置一致，Endpoints将pod组成一个列表
10 Name:              demo-nginx
11 Namespace:         default
12 Labels:            app=demo-nginx
13 Annotations:       <none>
14 Selector:          app=demo-nginx
15 Type:              ClusterIP        # service类型为ClusterIP
16 IP Family Policy:  SingleStack
17 IP Families:       IPv4
18 IP:                10.96.199.219
19 IPs:               10.96.199.219
20 Port:              <unset>  80/TCP    # 服务端口，即ClusterIP对外访问的端口
21 TargetPort:        80/TCP
22 Endpoints:         172.16.1.5:80,172.16.2.4:80    # 访问后端地址列表
23 Session Affinity:  None　　　　　　　　# 默认调度算法为：rr
24 Events:            <none>
25 [root@master1 ~]# 
26 [root@master1 ~]# kubectl get endpoints
27 NAME         ENDPOINTS                     AGE
28 demo-nginx   172.16.1.5:80,172.16.2.4:80   71s
29 kubernetes   192.168.1.66:6443             46h
30 [root@master1 ~]# 
31 [root@master1 ~]# kubectl describe endpoints demo-nginx 
32 Name:         demo-nginx
33 Namespace:    default
34 Labels:       app=demo-nginx
35 Annotations:  endpoints.kubernetes.io/last-change-trigger-time: 2021-10-03T08:59:17Z
36 Subsets:
37   Addresses:          172.16.1.5,172.16.2.4
38   NotReadyAddresses:  <none>
39   Ports:
40     Name     Port  Protocol
41     ----     ----  --------
42     <unset>  80    TCP
43 
44 Events:  <none>
45 [root@master1 ~]#

通过service访问服务，默认的调度算法是：rr；同时可以通过ipvsadm命令检查相关调度的详情：

 1 [root@master1 ~]# kubectl get service
 2 NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)   AGE
 3 demo-nginx   ClusterIP   10.96.199.219   <none>        80/TCP    6m52s
 4 kubernetes   ClusterIP   10.96.0.1       <none>        443/TCP   46h
 5 [root@master1 ~]# 
 6 [root@master1 ~]# curl http://10.96.199.219
 7 nginx-web1
 8 [root@master1 ~]# curl http://10.96.199.219
 9 nginx-web2
10 [root@master1 ~]# curl http://10.96.199.219
11 nginx-web1
12 [root@master1 ~]# curl http://10.96.199.219
13 nginx-web2
14 [root@master1 ~]# 
15 [root@master1 ~]# kubectl get pods -o wide
16 NAME                          READY   STATUS    RESTARTS   AGE   IP           NODE    NOMINATED NODE   READINESS GATES
17 demo-nginx-55ccc46db6-jmjkx   1/1     Running   0          52m   172.16.2.4   node2   <none>           <none>
18 demo-nginx-55ccc46db6-lg4nz   1/1     Running   0          52m   172.16.1.5   node1   <none>           <none>
19 [root@master1 ~]# 
20 [root@master1 ~]# ipvsadm -ln
21 IP Virtual Server version 1.2.1 (size=4096)
22 Prot LocalAddress:Port Scheduler Flags
23   -> RemoteAddress:Port           Forward Weight ActiveConn InActConn
24 TCP  10.96.0.1:443 rr
25   -> 192.168.1.66:6443            Masq    1      0          0         
26 TCP  10.96.0.10:53 rr
27   -> 172.16.0.4:53                Masq    1      0          0         
28   -> 172.16.0.5:53                Masq    1      0          0         
29 TCP  10.96.0.10:9153 rr
30   -> 172.16.0.4:9153              Masq    1      0          0         
31   -> 172.16.0.5:9153              Masq    1      0          0         
32 TCP  10.96.199.219:80 rr　　　　　　# 通过ipvsadm命令可以很清晰的看到想情况，包括上面访问了四次，两台服务器上各有两次访问
33   -> 172.16.1.5:80                Masq    1      0          2         
34   -> 172.16.2.4:80                Masq    1      0          2         
35 UDP  10.96.0.10:53 rr
36   -> 172.16.0.4:53                Masq    1      0          0         
37   -> 172.16.0.5:53                Masq    1      0          0         
38 [root@master1 ~]#

3.3、通过NodePort访问

　　Service通过ClusterIP只能提供集群内部的应用访问，外部无法直接访问应用，如果需要外部访问有如下几种方式：NodePort，LoadBalancer和Ingress，其中LoadBalancer需要由云服务提供商实现，Ingress需要安装单独的Ingress Controller，日常测试可以通过NodePort的方式实现，NodePort可以将node的某个端口暴露给外部网络访问。

修改type的类型由ClusterIP修改为NodePort类型（或者重新创建，指定type的类型为NodePort），并验证访问结果（包括原来的service的地址是否可以访问）

 1 [root@master1 ~]# kubectl get service demo-nginx -o yaml
 2 apiVersion: v1
 3 kind: Service
 4 metadata:
 5   creationTimestamp: "2021-10-03T08:59:17Z"
 6   labels:
 7     app: demo-nginx
 8   name: demo-nginx
 9   namespace: default
10   resourceVersion: "13905"
11   uid: 148f6bc4-963c-4751-a733-02fd8a88c0bc
12 spec:
13   clusterIP: 10.96.199.219
14   clusterIPs:
15   - 10.96.199.219
16   internalTrafficPolicy: Cluster
17   ipFamilies:
18   - IPv4
19   ipFamilyPolicy: SingleStack
20   ports:
21   - port: 80
22     protocol: TCP
23     targetPort: 80
24   selector:
25     app: demo-nginx
26   sessionAffinity: None
27   type: ClusterIP　　　　# 修改前
28 status:
29   loadBalancer: {}
30 [root@master1 ~]# 
31 [root@master1 ~]# kubectl patch service demo-nginx -p '{"spec":{"type":"NodePort"}}'        # 你也可以直接使用：kubectl edit service demo-nginx 直接修改
32 service/demo-nginx patched
33 [root@master1 ~]# 
34 [root@master1 ~]# kubectl get service demo-nginx -o yaml
35 apiVersion: v1
36 kind: Service
37 metadata:
38   creationTimestamp: "2021-10-03T08:59:17Z"
39   labels:
40     app: demo-nginx
41   name: demo-nginx
42   namespace: default
43   resourceVersion: "15278"
44   uid: 148f6bc4-963c-4751-a733-02fd8a88c0bc
45 spec:
46   clusterIP: 10.96.199.219
47   clusterIPs:
48   - 10.96.199.219
49   externalTrafficPolicy: Cluster
50   internalTrafficPolicy: Cluster
51   ipFamilies:
52   - IPv4
53   ipFamilyPolicy: SingleStack
54   ports:
55   - nodePort: 32638
56     port: 80
57     protocol: TCP
58     targetPort: 80
59   selector:
60     app: demo-nginx
61   sessionAffinity: None
62   type: NodePort　　　　# 修改后
63 status:
64   loadBalancer: {}
65 [root@master1 ~]# 
66 [root@master1 ~]# kubectl get service
67 NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
68 demo-nginx   NodePort    10.96.199.219   <none>        80:32638/TCP   16m
69 kubernetes   ClusterIP   10.96.0.1       <none>        443/TCP        46h
70 [root@master1 ~]# 
71 [root@master1 ~]# cat /etc/hosts
72 127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
73 ::1         localhost localhost.localdomain localhost6 localhost6.localdomain6
74 192.168.1.66    master1
75 192.168.1.77    node1
76 192.168.1.88    node2
77 [root@master1 ~]# # 验证访问结果：通过NodePort访问应用程序,每个node的地址相当于vip，可以实现相同的负载均衡效果，同时CluserIP功能依可用
78 [root@master1 ~]# curl http://master1:32638
79 nginx-web1
80 [root@master1 ~]# curl http://node1:32638
81 nginx-web1
82 [root@master1 ~]# curl http://node2:32638
83 nginx-web1
84 [root@master1 ~]# curl http://node2:32638
85 nginx-web2
86 [root@master1 ~]# # 验证访问结果：通过NodePort访问应用程序,同时CluserIP功能依可用 
87 [root@master1 ~]# curl http://10.96.199.219
88 nginx-web1
89 [root@master1 ~]# curl http://10.96.199.219
90 nginx-web2
91 [root@master1 ~]#

4、扩展应用

当应用程序的负载比较高无法满足应用请求时，一般我们会通过扩展RS的数量来实现，在kubernetes中，扩展RS实际上通过扩展副本数replicas来实现，扩展RS非常便利，快速实现弹性伸缩。kubernets能提供两种方式的伸缩能力：

手动伸缩能力scale up和scale down；
动态的弹性伸缩horizontalpodautoscalers,基于CPU的利用率实现自动的弹性伸缩，需要依赖与监控组件如metrics server，当前未实现，后续再做深入探讨，本文以手动的scale的方式扩展应用的副本数。

手动扩展副本数到四副本并查看：

 1 [root@master1 ~]# kubectl get deployments.apps  -o wide
 2 NAME         READY   UP-TO-DATE   AVAILABLE   AGE   CONTAINERS   IMAGES        SELECTOR
 3 demo-nginx   2/2     2            2           92m   nginx        nginx:1.8.1   app=demo-nginx
 4 [root@master1 ~]# 
 5 [root@master1 ~]# kubectl get deployments.apps  -o wide
 6 NAME         READY   UP-TO-DATE   AVAILABLE   AGE   CONTAINERS   IMAGES        SELECTOR
 7 demo-nginx   2/2     2            2           92m   nginx        nginx:1.8.1   app=demo-nginx
 8 [root@master1 ~]# 
 9 [root@master1 ~]# kubectl scale deployment demo-nginx --replicas=4　　　　#  扩容到4副本
10 deployment.apps/demo-nginx scaled
11 [root@master1 ~]# 
12 [root@master1 ~]# kubectl get deployments.apps  -o wide
13 NAME         READY   UP-TO-DATE   AVAILABLE   AGE   CONTAINERS   IMAGES        SELECTOR
14 demo-nginx   4/4     4            4           93m   nginx        nginx:1.8.1   app=demo-nginx
15 [root@master1 ~]#

此时再次查看service的情况会怎样呢？查看service详情,新扩展的pod会自动更新到service的endpoints中，自动服务发现：

 1 [root@master1 ~]# kubectl get service
 2 NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
 3 demo-nginx   NodePort    10.96.199.219   <none>        80:32638/TCP   51m
 4 kubernetes   ClusterIP   10.96.0.1       <none>        443/TCP        46h
 5 [root@master1 ~]# 
 6 [root@master1 ~]# kubectl describe service demo-nginx 
 7 Name:                     demo-nginx
 8 Namespace:                default
 9 Labels:                   app=demo-nginx
10 Annotations:              <none>
11 Selector:                 app=demo-nginx
12 Type:                     NodePort
13 IP Family Policy:         SingleStack
14 IP Families:              IPv4
15 IP:                       10.96.199.219
16 IPs:                      10.96.199.219
17 Port:                     <unset>  80/TCP
18 TargetPort:               80/TCP
19 NodePort:                 <unset>  32638/TCP
20 Endpoints:                172.16.1.5:80,172.16.1.6:80,172.16.2.4:80 + 1 more...    # 新生成的两个pod地址已经加进来了
21 Session Affinity:         None
22 External Traffic Policy:  Cluster
23 Events:                   <none>
24 [root@master1 ~]# 
25 [root@master1 ~]# kubectl describe endpoints demo-nginx 
26 Name:         demo-nginx
27 Namespace:    default
28 Labels:       app=demo-nginx
29 Annotations:  endpoints.kubernetes.io/last-change-trigger-time: 2021-10-03T09:47:26Z
30 Subsets:
31   Addresses:          172.16.1.5,172.16.1.6,172.16.2.4,172.16.2.5            # 新生成的两个pod地址已经加进来了
32   NotReadyAddresses:  <none>
33   Ports:
34     Name     Port  Protocol
35     ----     ----  --------
36     <unset>  80    TCP
37 
38 Events:  <none>
39 [root@master1 ~]#

测试验证，将新加入的两个pod站点内容设置为：web3，web4，参考前面的设置方法，测试service的ip，查看负载均衡效果：

 1 [root@master1 ~]# kubectl get pods -o wide
 2 NAME                          READY   STATUS    RESTARTS   AGE   IP           NODE    NOMINATED NODE   READINESS GATES
 3 demo-nginx-55ccc46db6-jmjkx   1/1     Running   0          93m   172.16.2.4   node2   <none>           <none>
 4 demo-nginx-55ccc46db6-l2vj5   1/1     Running   0          26s   172.16.2.5   node2   <none>           <none>
 5 demo-nginx-55ccc46db6-lg4nz   1/1     Running   0          93m   172.16.1.5   node1   <none>           <none>
 6 demo-nginx-55ccc46db6-vrlhw   1/1     Running   0          26s   172.16.1.6   node1   <none>           <none>
 7 [root@master1 ~]# # 修改新加入的两个pod的网页内容：
 8 [root@master1 ~]# kubectl exec -it demo-nginx-55ccc46db6-l2vj5 /bin/bash
 9 kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
10 root@demo-nginx-55ccc46db6-l2vj5:/# echo 'nginx-web3' > /usr/share/nginx/html/index.html
11 root@demo-nginx-55ccc46db6-l2vj5:/# exit
12 exit
13 [root@master1 ~]# kubectl exec -it demo-nginx-55ccc46db6-vrlhw /bin/bash
14 kubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.
15 root@demo-nginx-55ccc46db6-vrlhw:/# echo 'nginx-web4' > /usr/share/nginx/html/index.html
16 root@demo-nginx-55ccc46db6-vrlhw:/# exit
17 exit
18 [root@master1 ~]# 
19 [root@master1 ~]# kubectl get service
20 NAME         TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
21 demo-nginx   NodePort    10.96.199.219   <none>        80:32638/TCP   51m
22 kubernetes   ClusterIP   10.96.0.1       <none>        443/TCP        46h
23 [root@master1 ~]# # 验证情况如下：
24 [root@master1 ~]# curl http://10.96.199.219
25 nginx-web3
26 [root@master1 ~]# curl http://10.96.199.219
27 nginx-web4
28 [root@master1 ~]# curl http://10.96.199.219
29 nginx-web1
30 [root@master1 ~]# curl http://10.96.199.219
31 nginx-web2
32 [root@master1 ~]#

　　由此可知，弹性伸缩会自动自动加入到service中实现服务自动发现和负载均衡，应用的扩展相比于传统应用快速非常多。此外，kubernetes还支持自动弹性扩展的能力，即Horizontal Pod AutoScaler，自动横向伸缩能力，配合监控系统根据CPU的利用率弹性扩展Pod个数。

5、滚动升级

使用最新版本的nginx替换当前nginx:1.8.1的版本并检查验证相关结果：

 1 [root@master1 ~]# kubectl get deployments.apps demo-nginx -o wide
 2 NAME         READY   UP-TO-DATE   AVAILABLE   AGE    CONTAINERS   IMAGES        SELECTOR
 3 demo-nginx   4/4     4            4           106m   nginx        nginx:1.8.1   app=demo-nginx
 4 [root@master1 ~]# curl -I http://10.96.199.219
 5 HTTP/1.1 200 OK
 6 Server: nginx/1.8.1            # 升级前的nginx版本
 7 Date: Sun, 03 Oct 2021 10:01:58 GMT
 8 Content-Type: text/html
 9 Content-Length: 11
10 Last-Modified: Sun, 03 Oct 2021 09:49:12 GMT
11 Connection: keep-alive
12 ETag: "61597c98-b"
13 Accept-Ranges: bytes
14 
15 [root@master1 ~]# 
16 [root@master1 ~]# kubectl set image deployments.apps demo-nginx nginx=nginx:latest        # 具体的升级命令
17 deployment.apps/demo-nginx image updated
18 [root@master1 ~]# 
19 [root@master1 ~]# kubectl get deployments.apps demo-nginx -o wide
20 NAME         READY   UP-TO-DATE   AVAILABLE   AGE    CONTAINERS   IMAGES         SELECTOR
21 demo-nginx   4/4     4            4           110m   nginx        nginx:latest   app=demo-nginx
22 [root@master1 ~]# curl -I http://10.96.199.219
23 HTTP/1.1 200 OK
24 Server: nginx/1.21.3        # 升级后的nginx版本
25 Date: Sun, 03 Oct 2021 10:04:44 GMT
26 Content-Type: text/html
27 Content-Length: 615
28 Last-Modified: Tue, 07 Sep 2021 15:21:03 GMT
29 Connection: keep-alive
30 ETag: "6137835f-267"
31 Accept-Ranges: bytes
32 
33 [root@master1 ~]#

观察升级过程，通过查看可知，升级过程中是通过新建+删除的方式逐个替换pod的方式：

 1 [root@master1 ~]# kubectl get pods -w
 2 NAME                          READY   STATUS    RESTARTS   AGE
 3 demo-nginx-55ccc46db6-jmjkx   1/1     Running   0          107m
 4 demo-nginx-55ccc46db6-l2vj5   1/1     Running   0          13m
 5 demo-nginx-55ccc46db6-lg4nz   1/1     Running   0          107m
 6 demo-nginx-55ccc46db6-vrlhw   1/1     Running   0          13m
 7 # 这里是升级前后的分界……
 8 demo-nginx-6b99c5bdf5-qxdkh   0/1     Pending   0          0s
 9 demo-nginx-6b99c5bdf5-qxdkh   0/1     Pending   0          0s
10 demo-nginx-55ccc46db6-vrlhw   1/1     Terminating   0          16m
11 demo-nginx-6b99c5bdf5-qxdkh   0/1     ContainerCreating   0          0s
12 demo-nginx-6b99c5bdf5-vsrc8   0/1     Pending             0          0s
13 demo-nginx-6b99c5bdf5-vsrc8   0/1     Pending             0          0s
14 demo-nginx-6b99c5bdf5-vsrc8   0/1     ContainerCreating   0          0s
15 demo-nginx-55ccc46db6-vrlhw   0/1     Terminating         0          16m
16 demo-nginx-55ccc46db6-vrlhw   0/1     Terminating         0          16m
17 demo-nginx-55ccc46db6-vrlhw   0/1     Terminating         0          16m
18 demo-nginx-6b99c5bdf5-qxdkh   1/1     Running             0          2s
19 demo-nginx-55ccc46db6-l2vj5   1/1     Terminating         0          16m
20 demo-nginx-6b99c5bdf5-kwkzs   0/1     Pending             0          0s
21 demo-nginx-6b99c5bdf5-kwkzs   0/1     Pending             0          0s
22 demo-nginx-6b99c5bdf5-kwkzs   0/1     ContainerCreating   0          0s
23 demo-nginx-6b99c5bdf5-vsrc8   1/1     Running             0          2s
24 demo-nginx-55ccc46db6-lg4nz   1/1     Terminating         0          110m
25 demo-nginx-6b99c5bdf5-cpb2t   0/1     Pending             0          0s
26 demo-nginx-6b99c5bdf5-cpb2t   0/1     Pending             0          0s
27 demo-nginx-6b99c5bdf5-cpb2t   0/1     ContainerCreating   0          1s
28 demo-nginx-55ccc46db6-lg4nz   0/1     Terminating         0          110m
29 demo-nginx-55ccc46db6-lg4nz   0/1     Terminating         0          110m
30 demo-nginx-55ccc46db6-lg4nz   0/1     Terminating         0          110m
31 demo-nginx-55ccc46db6-l2vj5   0/1     Terminating         0          16m
32 demo-nginx-55ccc46db6-l2vj5   0/1     Terminating         0          16m
33 demo-nginx-55ccc46db6-l2vj5   0/1     Terminating         0          16m
34 demo-nginx-6b99c5bdf5-cpb2t   1/1     Running             0          3s
35 demo-nginx-55ccc46db6-jmjkx   1/1     Terminating         0          110m
36 demo-nginx-6b99c5bdf5-kwkzs   1/1     Running             0          3s
37 demo-nginx-55ccc46db6-jmjkx   0/1     Terminating         0          110m
38 demo-nginx-55ccc46db6-jmjkx   0/1     Terminating         0          110m
39 demo-nginx-55ccc46db6-jmjkx   0/1     Terminating         0          110m

再次查看deployments的详情可知道，deployments已经更换了新的replicasets，原来的replicasets的版本为1，可用于回滚；查看滚动升级的版本，可以看到有两个版本，分别对应的两个不同的replicasets：

 1 [root@master1 ~]# kubectl describe deployments.apps demo-nginx 
 2 Name:                   demo-nginx
 3 Namespace:              default
 4 CreationTimestamp:      Sun, 03 Oct 2021 04:13:57 -0400
 5 Labels:                 app=demo-nginx
 6 Annotations:            deployment.kubernetes.io/revision: 2    # 新的版本号，用于回滚
 7 Selector:               app=demo-nginx
 8 Replicas:               4 desired | 4 updated | 4 total | 4 available | 0 unavailable
 9 StrategyType:           RollingUpdate
10 MinReadySeconds:        0
11 RollingUpdateStrategy:  25% max unavailable, 25% max surge
12 Pod Template:
13   Labels:  app=demo-nginx
14   Containers:
15    nginx:
16     Image:        nginx:latest
17     Port:         <none>
18     Host Port:    <none>
19     Environment:  <none>
20     Mounts:       <none>
21   Volumes:        <none>
22 Conditions:
23   Type           Status  Reason
24   ----           ------  ------
25   Available      True    MinimumReplicasAvailable
26   Progressing    True    NewReplicaSetAvailable
27 OldReplicaSets:  <none>
28 NewReplicaSet:   demo-nginx-6b99c5bdf5 (4/4 replicas created)　　　　# 和下面的命令：kubectl get replicasets.apps查看结果对应上
29 Events:                                                            # replicasets 之间替换的过程
30   Type    Reason             Age    From                   Message
31   ----    ------             ----   ----                   -------
32   Normal  ScalingReplicaSet  23m    deployment-controller  Scaled up replica set demo-nginx-55ccc46db6 to 4
33   Normal  ScalingReplicaSet  6m45s  deployment-controller  Scaled up replica set demo-nginx-6b99c5bdf5 to 1
34   Normal  ScalingReplicaSet  6m45s  deployment-controller  Scaled down replica set demo-nginx-55ccc46db6 to 3
35   Normal  ScalingReplicaSet  6m45s  deployment-controller  Scaled up replica set demo-nginx-6b99c5bdf5 to 2
36   Normal  ScalingReplicaSet  6m43s  deployment-controller  Scaled down replica set demo-nginx-55ccc46db6 to 2
37   Normal  ScalingReplicaSet  6m43s  deployment-controller  Scaled up replica set demo-nginx-6b99c5bdf5 to 3
38   Normal  ScalingReplicaSet  6m43s  deployment-controller  Scaled down replica set demo-nginx-55ccc46db6 to 1
39   Normal  ScalingReplicaSet  6m43s  deployment-controller  Scaled up replica set demo-nginx-6b99c5bdf5 to 4
40   Normal  ScalingReplicaSet  6m40s  deployment-controller  Scaled down replica set demo-nginx-55ccc46db6 to 0
41 [root@master1 ~]# 
42 [root@master1 ~]# 
43 [root@master1 ~]# kubectl rollout history deployment demo-nginx 
44 deployment.apps/demo-nginx 
45 REVISION  CHANGE-CAUSE
46 1         <none>
47 2         <none>
48 
49 [root@master1 ~]# ：             # 查看replicasets列表,旧的包含pod为0
50 NAME                    DESIRED   CURRENT   READY   AGE
51 demo-nginx-55ccc46db6   0         0         0       127m
52 demo-nginx-6b99c5bdf5   4         4         4       17m
53 [root@master1 ~]#

回滚到旧的版本：

 1 [root@master1 ~]# kubectl rollout undo deployment demo-nginx --to-revision=1    # 回退到指定的版本
 2 deployment.apps/demo-nginx rolled back
 3 [root@master1 ~]# # 相关查看验证
 4 [root@master1 ~]# curl -I http://10.96.199.219
 5 HTTP/1.1 200 OK
 6 Server: nginx/1.8.1
 7 Date: Sun, 03 Oct 2021 10:27:31 GMT
 8 Content-Type: text/html
 9 Content-Length: 612
10 Last-Modified: Tue, 26 Jan 2016 15:24:47 GMT
11 Connection: keep-alive
12 ETag: "56a78fbf-264"
13 Accept-Ranges: bytes
14 
15 [root@master1 ~]# 
16 [root@master1 ~]# kubectl get replicasets.apps 
17 NAME                    DESIRED   CURRENT   READY   AGE
18 demo-nginx-55ccc46db6   4         4         4       133m
19 demo-nginx-6b99c5bdf5   0         0         0       23m
20 [root@master1 ~]# 
21 [root@master1 ~]# kubectl describe deployments.apps demo-nginx 
22 Name:                   demo-nginx
23 Namespace:              default
24 CreationTimestamp:      Sun, 03 Oct 2021 04:13:57 -0400
25 Labels:                 app=demo-nginx
26 Annotations:            deployment.kubernetes.io/revision: 3
27 Selector:               app=demo-nginx
28 Replicas:               4 desired | 4 updated | 4 total | 4 available | 0 unavailable
29 StrategyType:           RollingUpdate
30 MinReadySeconds:        0
31 RollingUpdateStrategy:  25% max unavailable, 25% max surge
32 Pod Template:
33   Labels:  app=demo-nginx
34   Containers:
35    nginx:
36     Image:        nginx:1.8.1
37     Port:         <none>
38     Host Port:    <none>
39     Environment:  <none>
40     Mounts:       <none>
41   Volumes:        <none>
42 Conditions:
43   Type           Status  Reason
44   ----           ------  ------
45   Available      True    MinimumReplicasAvailable
46   Progressing    True    NewReplicaSetAvailable
47 OldReplicaSets:  <none>
48 NewReplicaSet:   demo-nginx-55ccc46db6 (4/4 replicas created)
49 Events:
50   Type    Reason             Age                 From                   Message
51   ----    ------             ----                ----                   -------
52   Normal  ScalingReplicaSet  24m                 deployment-controller  Scaled up replica set demo-nginx-6b99c5bdf5 to 1
53   Normal  ScalingReplicaSet  24m                 deployment-controller  Scaled down replica set demo-nginx-55ccc46db6 to 3
54   Normal  ScalingReplicaSet  24m                 deployment-controller  Scaled up replica set demo-nginx-6b99c5bdf5 to 2
55   Normal  ScalingReplicaSet  23m                 deployment-controller  Scaled up replica set demo-nginx-6b99c5bdf5 to 3
56   Normal  ScalingReplicaSet  23m                 deployment-controller  Scaled down replica set demo-nginx-55ccc46db6 to 1
57   Normal  ScalingReplicaSet  23m                 deployment-controller  Scaled down replica set demo-nginx-55ccc46db6 to 2
58   Normal  ScalingReplicaSet  23m                 deployment-controller  Scaled up replica set demo-nginx-6b99c5bdf5 to 4
59   Normal  ScalingReplicaSet  23m                 deployment-controller  Scaled down replica set demo-nginx-55ccc46db6 to 0
60   Normal  ScalingReplicaSet  46s                 deployment-controller  Scaled up replica set demo-nginx-55ccc46db6 to 1
61   Normal  ScalingReplicaSet  46s                 deployment-controller  Scaled down replica set demo-nginx-6b99c5bdf5 to 3
62   Normal  ScalingReplicaSet  46s (x2 over 133m)  deployment-controller  Scaled up replica set demo-nginx-55ccc46db6 to 2
63   Normal  ScalingReplicaSet  43s (x2 over 40m)   deployment-controller  Scaled up replica set demo-nginx-55ccc46db6 to 4
64   Normal  ScalingReplicaSet  43s                 deployment-controller  Scaled down replica set demo-nginx-6b99c5bdf5 to 2
65   Normal  ScalingReplicaSet  43s                 deployment-controller  Scaled up replica set demo-nginx-55ccc46db6 to 3
66   Normal  ScalingReplicaSet  43s                 deployment-controller  Scaled down replica set demo-nginx-6b99c5bdf5 to 1
67   Normal  ScalingReplicaSet  41s                 deployment-controller  Scaled down replica set demo-nginx-6b99c5bdf5 to 0
68 [root@master1 ~]# kubectl rollout history deployment demo-nginx 
69 deployment.apps/demo-nginx 
70 REVISION  CHANGE-CAUSE
71 2         <none>
72 3         <none>
73 
74 [root@master1 ~]#

查看全文

相关阅读:
如何在Grid中做出快捷菜单效果？
ExtJs FormPanel布局
 wpf 中获取ComboBox中选定的文本值
 delphi中的dbgrid使用
 Delphi修改Access密码,压缩与修复,建立Access数据库文件
 关于OS X系统root账号的激活及密码更改
 Delphi过程函数传递参数的几种方式
 Eclipse里的快捷键
 Delphi封装类到DLL
Delphi Project 之工程选项（Project Options）

原文地址：https://www.cnblogs.com/zheng-weimin/p/15364320.html