二进制方式搭建Kubernetes 1.19.3高可用集群(五)——部署dashboard

二进制方式搭建Kubernetes 1.19.3高可用集群(五)——部署dashboard

本文将介绍在二进制部署的k8s集群中部署dashboar 2.0.4，并解决部署过程中metrics-server无法启动的问题

部署dashboard

首先，根据官方文档来，下载配置文件(官方文档地址：https://github.com/kubernetes/dashboard)

wget https://raw.githubusercontent.com/kubernetes/dashboard/v2.0.4/aio/deploy/recommended.yaml -O dashboard-deploy.yaml

由于镜像在国外，下载可能会比较慢，所有我把镜像放到了阿里云上，可以按需替换下镜像

kubernetesui/dashboard:v2.0.4 替换为 registry.cn-shanghai.aliyuncs.com/jieee/dashboard:v2.0.4
kubernetesui/metrics-scraper:v1.0.4 替换为 registry.cn-shanghai.aliyuncs.com/jieee/metrics-scraper:v1.0.4

然后直接部署

kubectl apply -f dashboard-deploy.yaml

# 检查pod和service(默认的namespace是kubernetes-dashboard)
kubectl get pod -n kubernetes-dashboard
#NAME                                         READY   STATUS    RESTARTS   AGE
#dashboard-metrics-scraper-7b59f7d4df-bj66m   1/1     Running   0          2m
#kubernetes-dashboard-7df8bc567d-slbhs        1/1     Running   0          2m

kubectl get svc -n kubernetes-dashboard
#NAME                        TYPE        CLUSTER-IP     EXTERNAL-IP   PORT(S)    AGE
#dashboard-metrics-scraper   ClusterIP   10.120.5.204    <none>        8000/TCP   2m
#kubernetes-dashboard        ClusterIP   10.120.209.68   <none>        443/TCP    2m

至此，dashboard就部署完成了，然后我们就可以在浏览器中输入地址访问了（我这里的地址是https://10.120.209.68）

注意：由于dashboard使用了自签证书，所有chrome浏览器可能无法访问，使用Firefox可以正常访问

生成TOKEN

打开网页后，需要我们登陆

支持dashboard支持2种方式登陆，一般我们选择使用Token方式，先来创建一个Service Account

dashboard-rbac.yaml(这里我直接赋予了cluster-admin角色)

apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: dashboard-admin
  namespace: kubernetes-dashboard
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: dashboard-admin-bind-cluster-role
  labels:
    k8s-app: kubernetes-dashboard
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: dashboard-admin
  namespace: kubernetes-dashboard

部署并生成token

kubectl apply -f dashboard-rbac.yaml

#获取TOKEN
kubectl -n kubernetes-dashboard describe secret $(kubectl -n kubernetes-dashboard get secret | grep dashboard-admin | awk '{print $1}')

#Name:         dashboard-admin-token-grxgp
#Namespace:    kubernetes-dashboard
#Labels:       <none>
#Annotations:  kubernetes.io/service-account.name: dashboard-admin
#              kubernetes.io/service-account.uid: 440d60e7-f75b-429f-ad2b-1a56d33e47c8
#
#Type:  kubernetes.io/service-account-token
#
#Data
#====
#ca.crt:     1363 bytes
#namespace:  20 bytes
#token:      eyJhbGciOiJSUzI1NiIsImtpZCI6InZmWF9vS29UWE53bVhKbkdUY3ZpLXdqYlBHc3VCUzdiamMzLS1FMDZhQUEifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlcm5ldGVzLWRhc2hib2FyZCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJkYXNoYm9hcmQtYWRtaW4tdG9rZW4tZ3J4Z3AiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoiZGFzaGJvYXJkLWFkbWluIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiNDQwZDYwZTctZjc1Yi00MjlmLWFkMmItMWE1NmQzM2U0N2M4Iiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50Omt1YmVybmV0ZXMtZGFzaGJvYXJkOmRhc2hib2FyZC1hZG1pbiJ9.Uerz4ERXLeyKDfuNW_l-K_3xr3lh4Iyc8B5U_TnW8tlWgrYAcijTF86QESprolDmhn7s7RqVwrfUAHvmKoI_d08ApTWouu1lnoGIsn-qUovYOtAnpr-sal4TTWu9tjScodqklOw1WrICUiUFxcEN1939ERqx2oESYiKUuT2yEt2stMGUmp02QkmyiYtfk5a6sZ14LcyLL_mtC09hF4vW4dz2_QdP3qVd6l-RHS5NDFnB4bBz8m6TG6h2kY09tiGcFgjNfkQhFdy6L0F_jczufj39MrcRWofxROGKNo_vq2sSidekODjpp6TAIF43k51gW9T_qhUnrflemJAbUseqnw

最后得到的一长串token就是登陆所需的token

登陆后就能看到整个集群的状态了

可是我们发现列表中，CPU和内存使用率都是空的，这是因为我们还没有安装metrics-server

安装metrics-server

官方文档：https://github.com/kubernetes-sigs/metrics-server

按照文档，我们先下载配置文件

wget https://github.com/kubernetes-sigs/metrics-server/releases/download/v0.3.7/components.yaml -O metrics-server.yaml

同样的，我们替换一下镜像地址

k8s.gcr.io/metrics-server/metrics-server:v0.3.7 替换为 registry.cn-shanghai.aliyuncs.com/jieee/metrics-server:v0.3.7

然后部署

kubectl apply -f metrics-server.yaml

#检查pod状态
kubectl get pod -n kube-system | grep metrics-server
# metrics-server-f964c4474-t5sx9             1/1     Running   0          2m

可以看到pod已经正常运行了。

然而，当我们回到dashboard中，发现CPU和内存信息还是没有出来，我们先来看一下pod日志

kubectl logs metrics-server-f964c4474-t5sx9 -n kube-system
#...
#E1107 05:15:45.224261       1 configmap_cafile_content.go:243] kube-system/extension-apiserver-authentication failed with : missing content for CA bundle "client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file"
#E1107 05:15:45.225200       1 configmap_cafile_content.go:243] key failed with : missing content for CA bundle "client-ca::kube-system::extension-apiserver-authentication::requestheader-client-ca-file"

我们发现了这两条错误日志，原来是我们部署apiserver是没有开启聚合功能，那我们就来开启一下吧

创建证书

cat > proxy-client-csr.json<<EOF
{
  "CN": "aggregator",
  "hosts": [],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "ST": "Hangzhou",
      "L": "Hangzhou",
      "O": "system:masters",
      "OU": "System"
    }
  ]
}
EOF

#创建证书
cfssl gencert -ca=../ca.pem -ca-key=../ca-key.pem -config=../ca-config.json -profile=kubernetes  proxy-client-csr.json | cfssljson -bare proxy-client

#分发证书至所有master节点
scp proxy-client*.pem root@10.0.50.101:/etc/kubernetes/pki/
scp proxy-client*.pem root@10.0.50.102:/etc/kubernetes/pki/
scp proxy-client*.pem root@10.0.50.103:/etc/kubernetes/pki/

 

修改apiserver的service文件

在启动命令中添加以下参数

vi /etc/systemd/system/kube-apiserver.service
...
  --proxy-client-cert-file=/etc/kubernetes/pki/proxy-client.pem 
  --proxy-client-key-file=/etc/kubernetes/pki/proxy-client-key.pem 
  --runtime-config=api/all=true 
  --requestheader-client-ca-file=/etc/kubernetes/pki/ca.pem 
  --requestheader-allowed-names=aggregator 
  --requestheader-extra-headers-prefix=X-Remote-Extra- 
  --requestheader-group-headers=X-Remote-Group 
  --requestheader-username-headers=X-Remote-User 
...

然后分别重启所有master节点的apiserver

systemctl daemon-reload && systemctl restart kube-apiserver

重建metrics-server

kubectl replace --force -f metrics-server.yaml

等待一段时间后，回到dashboard，刷新后发现 CPU和内存信息都出来了

同时，安装完metrics-server后，我们也可以在kubelet中使用metrics-server,如：

kubectl top node
#NAME                      CPU(cores)   CPU%   MEMORY(bytes)   MEMORY%
#kube-n-60-101.jieee.xyz   465m         11%    3503Mi          44%
#kube-n-60-102.jieee.xyz   257m         6%     2600Mi          33%
#kube-n-60-103.jieee.xyz   414m         10%    4092Mi          52%

证书配置

由于dashboard中使用了自签证书，导致chrome中无法访问，带来了一些不便，接下来我们为dashboard配置上证书

方式一：使用已有证书

先删除dashboard，然后修改配置文件

#删除
kubectl delete -f dashboard-deploy.yaml

#修改配置文件
vi dashboard-deploy.yaml
#找到以下内容，然后删除
---
apiVersion: v1
kind: Secret
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard-certs
  namespace: kubernetes-dashboard
type: Opaque

创建证书，可以通过阿里云申请1年免费证书，或者通过Let’s Encrypt生成90天免费证书，建免费证书存放在$HOME/certs目录下，取名为tls.crt和tls.key。

#创建证书
kubectl create secret generic kubernetes-dashboard-certs --from-file=$HOME/certs -n kubernetes-dashboard

#重新部署dashboard
kubectl apply -f dashboard-deploy.yaml

如此，证书就配置完成了

方式二：使用ingress

如果集群中已存在ingress，并且ingress配置了ssl（dashboard不支持http访问，所有必须支持ssl），那么可以用ingress卸载字签证书并替换成新证书。

ingress的部署可以查看Kubernetes使用Ingress nginx暴露服务并配置证书

配置文件：

cat > dashboard-ingress.yaml<<EOF
kind: Ingress
apiVersion: networking.k8s.io/v1
metadata:
  name: dashboard
  namespace: kubernetes-dashboard
  annotations:
    nginx.ingress.kubernetes.io/ssl-redirect: "true" # 强制跳转https
    nginx.ingress.kubernetes.io/rewrite-target: /
    nginx.ingress.kubernetes.io/secure-backends: "true"
    kubernetes.io/ingress.class: "nginx"
    nginx.ingress.kubernetes.io/backend-protocol: "HTTPS" # 代理后端https
spec:
  tls:
  - hosts:
    - '*.lingjie.tech'
    secretName: lingjie-tech
  rules:
  - host: dashboard.lingjie.tech
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: kubernetes-dashboard
            port:
              number: 443
EOF

#部署ingress
kubectl apply -f dashboard-ingress.yaml

然后我们绑定一下host（将dashboard.lingjie.tech绑定到ingress的service ip），就能通过https访问了。