一、Etcd数据库备份与恢复
Kubernetes 使用Etcd 数据库实时存储集群中的数据，安全起见，一定要备份！
1、kubeadm部署方式备份
①备份
ETCDCTL_API=3 etcdctl snapshot save snap.db --endpoints=https://127.0.0.1:2379 --cacert=/etc/kubernetes/pki/etcd/ca.crt --cert=/etc/kubernetes/pki/etcd/peer.crt --key=/etc/kubernetes/pki/etcd/peer.key
②恢复
暂停kube-apiserver和etcd容器
mv /etc/kubernetes/manifests /etc/kubernetes/manifests.bak mv /var/lib/etcd/ /var/lib/etcd.bak
执行恢复命令
ETCDCTL_API=3 etcdctl snapshot restore snap.db --data-dir=/var/lib/etcd
启动kube-apiserver和etcd容器
mv /etc/kubernetes/manifests.bak /etc/kubernetes/manifests
2、二进制部署方式备份
①备份
ETCDCTL_API=3 etcdctl snapshot save snap.db --endpoints=https://192.168.56.61:2379 --cacert=/opt/etcd/ssl/ca.pem --cert=/opt/etcd/ssl/server.pem --key=/opt/etcd/ssl/server-key.pem
②恢复
暂停kube-apiserver和etcd
systemctl stop kube-apiserver systemctl stop etcd mv /var/lib/etcd/default.etcd /var/lib/etcd/default.etcd.bak
在每个节点上恢复
ETCDCTL_API=3 etcdctl snapshot restore snap.db --name etcd-1 --initial-cluster="etcd-1=https://192.168.56.61:2380,etcd-2=https://192.168.56.62:2380,etcd-3=https://192.168.56.63:2380" --initial-cluster-token=etcd-cluster --initial-advertise-peer-urls=https://192.168.56.61:2380 --data-dir=/var/lib/etcd/default.etcd
启动kube-apiserver和etcd服务
systemctl start kube-apiserver
systemctl start etcd
二、Node节点扩容
1、Bootstrap Token 方式增加Node节点
在kubernetes集群中，Node上组件kubelet和kube-proxy都需要与kube-apiserver进行通信，为了增加传输安全性，采用https方式。这就涉及到Node组件需要具备kube-apiserver用的证书颁发机构（CA）签发客户端证书，当规模较大时，这种客户端证书颁发需要大量工作，同样也会增加集群扩展复杂度。
为了简化流程，Kubernetes引入了TLS bootstraping机制来自动颁发客户端证书，所以强烈建议在Node上使用这种方式。
①kube-apiserver配置文件中是否启用Bootstrap Token
参与官方文档
https://kubernetes.io/zh/docs/reference/access-authn-authz/bootstrap-tokens/
https://kubernetes.io/zh/docs/reference/command-line-tools-reference/kubelet-tls-bootstrapping/
[root@k8s-node1 ~]# cat /app/kubernetes/cfg/kube-apiserver.conf KUBE_APISERVER_OPTS="--logtostderr=false --v=2 --log-dir=/app/kubernetes/logs --etcd-servers=https://192.168.29.15:2379,https://192.168.29.16:2379,https://192.168.29.17:2379 --bind-address=192.168.29.15 --secure-port=6443 --advertise-address=192.168.29.15 --allow-privileged=true --service-cluster-ip-range=10.0.0.0/24 --enable-admission-plugins=NamespaceLifecycle,LimitRanger,ServiceAccount,ResourceQuota,NodeRestriction --authorization-mode=RBAC,Node --enable-bootstrap-token-auth=true --token-auth-file=/app/kubernetes/cfg/token.csv
②使用Secret存储Bootstrap Token
Bootstrap Token值格式：07401b.f395accd246ae52d (点左边是Token ID，右边Token Secret)
apiVersion: v1 kind: Secret metadata: # name 必须是 "bootstrap-token-<token id>" 格式的 name: bootstrap-token-07401b namespace: kube-system # type 必须是 'bootstrap.kubernetes.io/token' type: bootstrap.kubernetes.io/token stringData: # 供人阅读的描述，可选。 description: "The default bootstrap token generated by 'kubeadm init'." # 令牌 ID 和秘密信息，必需。 token-id: 07401b token-secret: base64(f395accd246ae52d) # 可选的过期时间字段 expiration: "2025-10-10T03:22:11Z" #只修改此处即可 # 允许的用法 usage-bootstrap-authentication: "true" usage-bootstrap-signing: "true" # 令牌要认证为的额外组，必须以 "system:bootstrappers:" 开头 auth-extra-groups: system:bootstrappers:worker,system:bootstrappers:ingress
创建bootstrap-token.yaml即可
③创建RBAC角色绑定，允许kubelet tls bootstrap创建CSR请求
# Approve all CSRs for the group "system:bootstrappers" kind: ClusterRoleBinding apiVersion: rbac.authorization.k8s.io/v1 metadata: name: auto-approve-csrs-for-group subjects: - kind: Group name: system:bootstrappers apiGroup: rbac.authorization.k8s.io roleRef: kind: ClusterRole name: approve-node-client-csr apiGroup: rbac.authorization.k8s.io
创建bootstrap-rbac.yaml
④node节点安装组件，Docker、kubelet、kube-proxy、cni插件等
修改kubelet、kube-proxy配置文件中主机名
⑤kubelet配置Bootstrap kubeconfig文件
[root@k8s-node1 ~]# cat /app/kubernetes/cfg/kubelet.conf KUBELET_OPTS="--logtostderr=false --v=4 --log-dir=/app/kubernetes/logs --hostname-override=k8s-node1 --network-plugin=cni --kubeconfig=/app/kubernetes/cfg/kubelet.kubeconfig --bootstrap-kubeconfig=/app/kubernetes/cfg/bootstrap.kubeconfig --config=/app/kubernetes/cfg/kubelet-config.yml --cert-dir=/app/kubernetes/ssl --pod-infra-container-image=lizhenliang/pause-amd64:3.0"