Kubernetes实战总结

Kubernetes 需要 PKI 证书才能进行基于 TLS 的身份验证，如果你是使用kubeadm安装的 Kubernetes，则会自动生成集群所需的证书。

♦  API 服务器端点的证书

♦  Kubelet 的客户端证书，用于 API 服务器身份验证

♦  集群管理员的客户端证书，用于 API 服务器身份认证

♦  API 服务器的客户端证书，用于和 Kubelet 的会话

♦  API 服务器的客户端证书，用于和 etcd 的会话

♦  控制管理器的客户端证书/kubeconfig，用于和 API server 的会话

♦  调度器的客户端证书/kubeconfig，用于和 API server 的会话

♦  前端代理的客户端及服务端证书

详情参考官方说明：https://kubernetes.io/zh/docs/setup/best-practices/certificates/ 

---

查看证书

注意：默认根证书有效期为10年，其他所有证书有效期为1年。

[root@ymt108 ~]# cd /etc/kubernetes/pki
[root@ymt108 pki]# tree
.
├── apiserver.crt
├── apiserver-etcd-client.crt
├── apiserver-etcd-client.key
├── apiserver.key
├── apiserver-kubelet-client.crt
├── apiserver-kubelet-client.key
├── ca.crt
├── ca.key
├── etcd
│   ├── ca.crt
│   ├── ca.key
│   ├── healthcheck-client.crt
│   ├── healthcheck-client.key
│   ├── peer.crt
│   ├── peer.key
│   ├── server.crt
│   └── server.key
├── front-proxy-ca.crt
├── front-proxy-ca.key
├── front-proxy-client.crt
├── front-proxy-client.key
├── sa.key
└── sa.pub

1 directory, 22 files
[root@ymt108 pki]# kubeadm alpha certs check-expiration
[check-expiration] Reading configuration from the cluster...
[check-expiration] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'

CERTIFICATE                EXPIRES                  RESIDUAL TIME   CERTIFICATE AUTHORITY   EXTERNALLY MANAGED
admin.conf                 Jul 03, 2021 01:02 UTC   322d                                    no
apiserver                  Jul 03, 2021 01:02 UTC   322d            ca                      no
apiserver-etcd-client      Jul 03, 2021 01:02 UTC   322d            etcd-ca                 no
apiserver-kubelet-client   Jul 03, 2021 01:02 UTC   322d            ca                      no
controller-manager.conf    Jul 03, 2021 01:02 UTC   322d                                    no
etcd-healthcheck-client    Jul 03, 2021 01:02 UTC   322d            etcd-ca                 no
etcd-peer                  Jul 03, 2021 01:02 UTC   322d            etcd-ca                 no
etcd-server                Jul 03, 2021 01:02 UTC   322d            etcd-ca                 no
front-proxy-client         Jul 03, 2021 01:02 UTC   322d            front-proxy-ca          no
scheduler.conf             Jul 03, 2021 01:02 UTC   322d                                    no

CERTIFICATE AUTHORITY   EXPIRES                  RESIDUAL TIME   EXTERNALLY MANAGED
ca                      Jul 01, 2030 01:02 UTC   9y              no
etcd-ca                 Jul 01, 2030 01:02 UTC   9y              no
front-proxy-ca          Jul 01, 2030 01:02 UTC   9y              no

 

---

更新证书

1、手动生成证书自定义时长

　　　　我们可以通过 easyrsa、openssl 或 cfssl 手动地为集群生成证书，然后自定义证书时长。

　　　　详情参考官方说明：https://kubernetes.io/zh/docs/concepts/cluster-administration/certificates/

　　

2、定期升级集群来升级证书时长

　　　　kubeadm 会在控制面板升级的时候更新所有证书，这个功能旨在解决最简单的用例。

　　　　详情参考官方说明：https://kubernetes.cn/zh/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade/

• 如果你对此类证书的更新没有特殊要求，并且定期执行 Kubernetes 版本升级（每次升级之间的间隔时间少于 1 年），则 kubeadm 将确保你的集群保持最新状态并保持合理的安全性。
• 如果你对证书更新有更复杂的需求，则可通过将 --certificate-renewal=false 传递给 kubeadm upgrade apply 或者 kubeadm upgrade node ，从而选择不采用默认行为。

3、通过kubeadm命令升级证书时长

你可以随时通过 kubeadm alpha certs renew 命令手动更新证书，也可以选择更新单个证书或者全部证书。

流程如下：获取集群配置  ->  依据集群配置升级证书  ->  查看证书  ->  重启k8s容器

 注意： 如果你运行了一个 HA 集群，这个命令需要在所有控制面板节点上执行。

[root@k8s-32 ~]# kubeadm config view > kubeadm-config.yaml
[root@k8s-32 ~]# cat kubeadm-config.yaml
apiServer:
  extraArgs:
    authorization-mode: Node,RBAC
  timeoutForControlPlane: 4m0s
apiVersion: kubeadm.k8s.io/v1beta2
certificatesDir: /etc/kubernetes/pki
clusterName: kubernetes
controllerManager: {}
dns:
  type: CoreDNS
etcd:
  local:
    dataDir: /var/lib/etcd
imageRepository: registry.aliyuncs.com/google_containers
kind: ClusterConfiguration
kubernetesVersion: v1.17.5
networking:
  dnsDomain: cluster.local
  podSubnet: 10.11.0.0/16
  serviceSubnet: 10.96.0.0/12
scheduler: {}
[root@k8s-32 ~]# kubeadm alpha certs renew all --config=kubeadm-config.yaml
W0814 13:52:02.107428   26087 validation.go:28] Cannot validate kube-proxy config - no validator is available
W0814 13:52:02.107499   26087 validation.go:28] Cannot validate kubelet config - no validator is available
certificate embedded in the kubeconfig file for the admin to use and for kubeadm itself renewed
certificate for serving the Kubernetes API renewed
certificate the apiserver uses to access etcd renewed
certificate for the API server to connect to kubelet renewed
certificate embedded in the kubeconfig file for the controller manager to use renewed
certificate for liveness probes to healthcheck etcd renewed
certificate for etcd nodes to communicate with each other renewed
certificate for serving etcd renewed
certificate for the front proxy client renewed
certificate embedded in the kubeconfig file for the scheduler manager to use renewed
[root@k8s-32 ~]# kubeadm alpha certs check-expiration
[check-expiration] Reading configuration from the cluster...
[check-expiration] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'

CERTIFICATE                EXPIRES                  RESIDUAL TIME   CERTIFICATE AUTHORITY   EXTERNALLY MANAGED
admin.conf                 Aug 14, 2021 05:52 UTC   364d                                    no
apiserver                  Aug 14, 2021 05:52 UTC   364d            ca                      no
apiserver-etcd-client      Aug 14, 2021 05:52 UTC   364d            etcd-ca                 no
apiserver-kubelet-client   Aug 14, 2021 05:52 UTC   364d            ca                      no
controller-manager.conf    Aug 14, 2021 05:52 UTC   364d                                    no
etcd-healthcheck-client    Aug 14, 2021 05:52 UTC   364d            etcd-ca                 no
etcd-peer                  Aug 14, 2021 05:52 UTC   364d            etcd-ca                 no
etcd-server                Aug 14, 2021 05:52 UTC   364d            etcd-ca                 no
front-proxy-client         Aug 14, 2021 05:52 UTC   364d            front-proxy-ca          no
scheduler.conf             Aug 14, 2021 05:52 UTC   364d                                    no

CERTIFICATE AUTHORITY   EXPIRES                  RESIDUAL TIME   EXTERNALLY MANAGED
ca                      Jun 18, 2030 10:08 UTC   9y              no
etcd-ca                 Jun 18, 2030 10:08 UTC   9y              no
front-proxy-ca          Jun 18, 2030 10:08 UTC   9y              no
[root@k8s-32 ~]# docker ps |grep -E 'k8s_kube-apiserver|k8s_kube-controller-manager|k8s_kube-scheduler|k8s_etcd_etcd' | awk -F ' ' '{print $1}' |xargs docker restart
5241a2c98b05
5daf6684f286
a9b9e4798650
a329037569f6

4、编译kubeadm源码自定义证书时长　

# 下载kubernetes源码
wget https://github.com/kubernetes/kubernetes/archive/v1.17.5.tar.gz
tar -zxvf v1.17.5.tar.gz
# 修改证书时间
vim kubernetes-1.17.5/cmd/kubeadm/app/constants/constants.go
……
37 const (
……
46     // CertificateValidity defines the validity for all the signed certificates generated by kubeadm
47     CertificateValidity = time.Hour * 24 * 365 * 10
……
# 安装GO相关组件
yum -y install gcc make rsync jq
wget https://dl.google.com/go/go1.13.9.linux-amd64.tar.gz
tar -zxvf go1.13.9.linux-amd64.tar.gz -C /usr/local/
# 配置环境变量
vim /etc/profile
……
export GO_HOME=/usr/local/go
export PATH=$PATH:$GO_HOME/bin
……
source /etc/profile
# 构建kubeadm
cd kubernetes-1.17.5
make all WHAT=cmd/kubeadm GOFLAGS=-v
cp _output/local/bin/linux/amd64/kubeadm /usr/bin/kubeadm

 　  个人想法： 手动生成证书和编译kubeadm都有点繁琐，定期升级k8s版本也得要项目情况，最简单就是每年进行一次 kubeadm alpha certs renew 即可。

参考博文： 使用 kubeadm 进行证书管理          附025.kubeadm部署Kubernetes更新证书

作者：Leozhanggg

出处：https://www.cnblogs.com/leozhanggg/p/13401877.html

本文版权归作者和博客园共有，欢迎转载，但未经作者同意必须保留此段声明，且在文章页面明显位置给出原文连接，否则保留追究法律责任的权利。