升级
先从负载均衡服务器上面禁用，防止调度到正在做升级相应服务器上 (记得，升级完后要把注释去掉，重新加载haproxy)
[root@haproxy ~]# cat /etc/haproxy/haproxy.cfg
listen k8s-m44-6443
bind 172.18.8.168:6443
mode tcp
#server 172.18.8.109 172.18.8.109:6443 check inter 2s fall 3 rise 5
server 172.18.8.119 172.18.8.119:6443 check inter 2s fall 3 rise 5
server 172.18.8.129 172.18.8.129:6443 check inter 2s fall 3 rise 5
重新加载haproxy
[root@haproxy ~]# systemctl reload haproxy.service
先安装指定版本（每个master都执行）
[root@k8s-server1 ~]# apt-cache madison kubeadm
[root@k8s-server1 ~]# apt install kubeadm=1.20.9-00 kubelet=1.20.9-00 kubectl=1.20.9-00
...
[y/n] y
查看升级计划
[root@k8s-server1 manifests]# kubeadm upgrade plan
[upgrade/config] Making sure the configuration is correct:
[upgrade/config] Reading configuration from the cluster...
[upgrade/config] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
[preflight] Running pre-flight checks.
[upgrade] Running cluster health checks
[upgrade] Fetching available versions to upgrade to
[upgrade/versions] Cluster version: v1.20.5
[upgrade/versions] kubeadm version: v1.20.9
I0727 07:39:27.335261 26453 version.go:254] remote version is much newer: v1.21.3; falling back to: stable-1.20
[upgrade/versions] Latest stable version: v1.20.9
[upgrade/versions] Latest stable version: v1.20.9
[upgrade/versions] Latest version in the v1.20 series: v1.20.9
[upgrade/versions] Latest version in the v1.20 series: v1.20.9
Components that must be upgraded manually after you have upgraded the control plane with 'kubeadm upgrade apply':
COMPONENT CURRENT AVAILABLE
kubelet 6 x v1.20.5 v1.20.9
Upgrade to the latest version in the v1.20 series:
COMPONENT CURRENT AVAILABLE
kube-apiserver v1.20.5 v1.20.9
kube-controller-manager v1.20.5 v1.20.9
kube-scheduler v1.20.5 v1.20.9
kube-proxy v1.20.5 v1.20.9
CoreDNS 1.7.0 1.7.0
etcd 3.4.13-0 3.4.13-0
You can now apply the upgrade by executing the following command:
kubeadm upgrade apply v1.20.9
_____________________________________________________________________
The table below shows the current state of component configs as understood by this version of kubeadm.
Configs that have a "yes" mark in the "MANUAL UPGRADE REQUIRED" column require manual config upgrade or
resetting to kubeadm defaults before a successful upgrade can be performed. The version to manually
upgrade to is denoted in the "PREFERRED VERSION" column.
API GROUP CURRENT VERSION PREFERRED VERSION MANUAL UPGRADE REQUIRED
kubeproxy.config.k8s.io v1alpha1 v1alpha1 no
kubelet.config.k8s.io v1beta1 v1beta1 no
_____________________________________________________________________
执行（每个master）
[root@k8s-server1 ~]# kubeadm upgrade apply v1.20.9
[y/n] y
...
[upgrade/successful] SUCCESS! Your cluster was upgraded to "v1.20.9". Enjoy!
node节点执行（每个node节点）
[root@k8s-node1 ~]# apt install kubeadm=1.20.9-00 kubelet=1.20.9-00
...
[y/n] y
查看版本是否有变化
[root@k8s-server1 ~]# kubectl get node
NAME STATUS ROLES AGE VERSION
k8s-node1 Ready <none> 2d3h v1.20.9
k8s-node2 Ready <none> 2d3h v1.20.9
k8s-node3 Ready <none> 43h v1.20.9
k8s-server1 Ready control-plane,master 2d4h v1.20.9
k8s-server2 Ready control-plane,master 2d3h v1.20.9
k8s-server3 Ready control-plane,master 2d2h v1.20.9
降级
先下载比上个版本低的版本号
[root@k8s-server1 ~]# apt install kubeadm=1.20.5-00 kubelet=1.20.5-00 kubectl=1.20.5-00
...
[y/n] y
查看计划
[root@k8s-server1 ~]# kubeadm upgrade plan
执行
[root@k8s-server1 ~]# kubeadm upgrade apply v1.20.5
...
[y/n] y
...
[upgrade/successful] SUCCESS! Your cluster was upgraded to "v1.20.5". Enjoy!
node节点执行
[root@k8s-node1 ~]# apt install kubeadm=1.20.5-00 kubelet=1.20.5-00
...
[y/n] y
查看
[root@k8s-server1 ~]# kubectl get node
NAME STATUS ROLES AGE VERSION
k8s-node1 Ready <none> 2d3h v1.20.5
k8s-node2 Ready <none> 2d3h v1.20.5
k8s-node3 Ready <none> 43h v1.20.5
k8s-server1 Ready control-plane,master 2d4h v1.20.5
k8s-server2 Ready control-plane,master 2d3h v1.20.5
k8s-server3 Ready control-plane,master 2d3h v1.20.5