1、什么是K8s的弹性伸缩？
答：Hpa（全称叫做Horizontal Pod Autoscaler），Horizontal Pod Autoscaler的操作对象是Replication Controller、ReplicaSet或者Deployment对应的Pod（k8s中可以控制Pod的是rc、rs、deployment），根据观察到的CPU使用量与用户的阈值进行比对，做出是否需要增加或者减少实例数量的决策。controller目前使用heapSter来检测CPU使用量，检测周期默认是30秒。
2、K8s的弹性伸缩的工作原理？
答：Horizontal Pod Autoscaler的工作原理，主要是监控一个Pod，监控这个Pod的资源CPU使用率，一旦达到了设置的阈值，就做策略来决定它是否需要增加，做策略的时候还需要一个周期，比如，持续五分钟都发现CPU使用率高，就抓紧增加Pod的数量来减轻它的压力。当然也有一个策略，就是持续五分钟之后，压力一直都很低，那么会减少Pod的数量。这就是k8s的弹性伸缩的工作原理，主要是监控CPU的使用率，然后来决定是否增加或者减少Pod的数量。
3、K8s的弹性伸缩的实践，为了演示效果，这里对rc进行cpu资源的进行限制，方便压力测试效果。
1 [root@k8s-master ~]# cd k8s/ 2 [root@k8s-master k8s]# ls 3 book-master.war dashboard dashboard.zip deploy health heapster namespace pod rc skydns skydns.zip svc tomcat_demo tomcat_demo.zip 4 [root@k8s-master k8s]# mkdir hpa 5 [root@k8s-master k8s]# cd hpa/ 6 [root@k8s-master hpa]# ls 7 [root@k8s-master hpa]# cp ../rc/nginx_rc.yaml . 8 [root@k8s-master hpa]# ls 9 nginx_rc.yaml 10 [root@k8s-master hpa]# vim nginx_rc.yaml
配置内容，如下所示：
1 apiVersion: v1 2 kind: Pod 3 metadata: 4 name: myweb 5 labels: 6 app: web 7 env: myweb 8 spec: 9 containers: 10 - name: myweb 11 image: 192.168.110.133:5000/nginx:1.13 12 ports: 13 - containerPort: 80 14 resources: 15 # 最大可以使用的资源，100m的cpu时间片，50Mi的内存。 16 limits: 17 cpu: 100m 18 memory: 50Mi 19 # requests代表资源Pod需求的资源，100m的cpu时间片，50Mi的内存。 20 requests: 21 cpu: 100m 22 memory: 50Mi
开始创建这个RC，如下所示：
1 [root@k8s-master hpa]# kubectl create -f nginx_rc.yaml 2 replicationcontroller "myweb" created 3 [root@k8s-master hpa]#
查看初始的数量是两个，如下所示：
1 [root@k8s-master hpa]# kubectl get all 2 NAME DESIRED CURRENT READY AGE 3 rc/myweb 2 2 2 31s 4 5 NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE 6 svc/kubernetes 10.254.0.1 <none> 443/TCP 23d 7 8 NAME READY STATUS RESTARTS AGE 9 po/myweb-c0rs7 1/1 Running 0 31s 10 po/myweb-jkqc7 1/1 Running 0 31s 11 [root@k8s-master hpa]# kubectl get all -o wide 12 NAME DESIRED CURRENT READY AGE CONTAINER(S) IMAGE(S) SELECTOR 13 rc/myweb 2 2 2 35s myweb 192.168.110.133:5000/nginx:1.13 app=myweb 14 15 NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR 16 svc/kubernetes 10.254.0.1 <none> 443/TCP 23d <none> 17 18 NAME READY STATUS RESTARTS AGE IP NODE 19 po/myweb-c0rs7 1/1 Running 0 35s 172.16.16.4 k8s-node3 20 po/myweb-jkqc7 1/1 Running 0 35s 172.16.94.3 k8s-node2 21 [root@k8s-master hpa]#
4、开始创建hpa，可以通过配置文件或者命令进行声明，如下所示：
1 kubectl autoscale 资源类型（rc、deployment） 资源的名称 最大的pod数量 最小的pod数量 pod设定的阈值，cpu达到%多少使用率的时候就会触发hpa。
pod设定的阈值，cpu达到%多少使用率的时候就会触发hpa，这里测试的时候设置的比较低，生产环境可以设置60%、70%这些，较高些。
1 [root@k8s-master hpa]# kubectl autoscale replicationcontroller myweb --max=8 --min=1 --cpu-percent=5 2 replicationcontroller "myweb" autoscaled 3 [root@k8s-master hpa]#
此时，查看启动的资源，如下所示：
1 [root@k8s-master hpa]# kubectl get all 2 NAME REFERENCE TARGET CURRENT MINPODS MAXPODS AGE 3 hpa/myweb ReplicationController/myweb 5% <waiting> 1 8 53s 4 5 NAME DESIRED CURRENT READY AGE 6 rc/myweb 2 2 2 7m 7 8 NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE 9 svc/kubernetes 10.254.0.1 <none> 443/TCP 23d 10 11 NAME READY STATUS RESTARTS AGE 12 po/myweb-c0rs7 1/1 Running 0 7m 13 po/myweb-jkqc7 1/1 Running 0 7m 14 [root@k8s-master hpa]# kubectl get all -o wide 15 NAME REFERENCE TARGET CURRENT MINPODS MAXPODS AGE 16 hpa/myweb ReplicationController/myweb 5% <waiting> 1 8 56s 17 18 NAME DESIRED CURRENT READY AGE CONTAINER(S) IMAGE(S) SELECTOR 19 rc/myweb 2 2 2 7m myweb 192.168.110.133:5000/nginx:1.13 app=myweb 20 21 NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR 22 svc/kubernetes 10.254.0.1 <none> 443/TCP 23d <none> 23 24 NAME READY STATUS RESTARTS AGE IP NODE 25 po/myweb-c0rs7 1/1 Running 0 7m 172.16.16.4 k8s-node3 26 po/myweb-jkqc7 1/1 Running 0 7m 172.16.94.3 k8s-node2 27 [root@k8s-master hpa]#
这里可以看到由hpa控制rc，rc来控制pod的数量，现在开始进行压力测试，这里使用的ab的命令，首先查询到这个pod的ip地址，如下所示：
1 [root@k8s-master hpa]# kubectl get all -o wide 2 NAME REFERENCE TARGET CURRENT MINPODS MAXPODS AGE 3 hpa/myweb ReplicationController/myweb 5% <waiting> 1 8 3m 4 5 NAME DESIRED CURRENT READY AGE CONTAINER(S) IMAGE(S) SELECTOR 6 rc/myweb 2 2 2 10m myweb 192.168.110.133:5000/nginx:1.13 app=myweb 7 8 NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR 9 svc/kubernetes 10.254.0.1 <none> 443/TCP 23d <none> 10 11 NAME READY STATUS RESTARTS AGE IP NODE 12 po/myweb-c0rs7 1/1 Running 0 10m 172.16.16.4 k8s-node3 13 po/myweb-jkqc7 1/1 Running 0 10m 172.16.94.3 k8s-node2 14 [root@k8s-master hpa]#
首先安装一下httpd，安装了这个就包含了ab的命令了，如下所示：
1 [root@k8s-master ~]# yum install httpd-tools.x86_64 -y 2 Loaded plugins: fastestmirror, langpacks, product-id, search-disabled-repos, subscription-manager 3 4 This system is not registered with an entitlement server. You can use subscription-manager to register. 5 6 Determining fastest mirrors 7 * base: mirrors.tuna.tsinghua.edu.cn 8 * extras: mirrors.bfsu.edu.cn 9 * updates: mirrors.bfsu.edu.cn 10 base | 3.6 kB 00:00:00 11 extras | 2.9 kB 00:00:00 12 updates | 2.9 kB 00:00:00 13 updates/7/x86_64/primary_db | 2.9 MB 00:00:03 14 Resolving Dependencies 15 --> Running transaction check 16 ---> Package httpd-tools.x86_64 0:2.4.6-93.el7.centos will be installed 17 --> Processing Dependency: libaprutil-1.so.0()(64bit) for package: httpd-tools-2.4.6-93.el7.centos.x86_64 18 --> Processing Dependency: libapr-1.so.0()(64bit) for package: httpd-tools-2.4.6-93.el7.centos.x86_64 19 --> Running transaction check 20 ---> Package apr.x86_64 0:1.4.8-5.el7 will be installed 21 ---> Package apr-util.x86_64 0:1.5.2-6.el7 will be installed 22 --> Finished Dependency Resolution 23 24 Dependencies Resolved 25 26 ================================================================================================================================================================================================================= 27 Package Arch Version Repository Size 28 ================================================================================================================================================================================================================= 29 Installing: 30 httpd-tools x86_64 2.4.6-93.el7.centos base 92 k 31 Installing for dependencies: 32 apr x86_64 1.4.8-5.el7 base 103 k 33 apr-util x86_64 1.5.2-6.el7 base 92 k 34 35 Transaction Summary 36 ================================================================================================================================================================================================================= 37 Install 1 Package (+2 Dependent packages) 38 39 Total download size: 288 k 40 Installed size: 584 k 41 Downloading packages: 42 (1/3): httpd-tools-2.4.6-93.el7.centos.x86_64.rpm | 92 kB 00:00:00 43 (2/3): apr-util-1.5.2-6.el7.x86_64.rpm | 92 kB 00:00:00 44 (3/3): apr-1.4.8-5.el7.x86_64.rpm | 103 kB 00:00:00 45 ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- 46 Total 273 kB/s | 288 kB 00:00:01 47 Running transaction check 48 Running transaction test 49 Transaction test succeeded 50 Running transaction 51 Installing : apr-1.4.8-5.el7.x86_64 1/3 52 Installing : apr-util-1.5.2-6.el7.x86_64 2/3 53 Installing : httpd-tools-2.4.6-93.el7.centos.x86_64 3/3 54 Verifying : apr-1.4.8-5.el7.x86_64 1/3 55 Verifying : httpd-tools-2.4.6-93.el7.centos.x86_64 2/3 56 Verifying : apr-util-1.5.2-6.el7.x86_64 3/3 57 58 Installed: 59 httpd-tools.x86_64 0:2.4.6-93.el7.centos 60 61 Dependency Installed: 62 apr.x86_64 0:1.4.8-5.el7 apr-util.x86_64 0:1.5.2-6.el7 63 64 Complete! 65 [root@k8s-master ~]#
压力测试，总共发起500000次请求，每次发起30个请求，如下所示：
1 [root@k8s-master ~]# ab -n 500000 -c 30 http://172.16.94.3/index.html/
Kubernetes dashboard的界面，现在，如下所示：
由于我的是启动了两个Pod，这里我也同时压力测试两个Pod，第二个Pod的压力测试如下所示：
1 [root@k8s-master hpa]# ab -n 500000 -c 30 http://172.16.16.4/index.html/ 2 This is ApacheBench, Version 2.3 <$Revision: 1430300 $> 3 Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/ 4 Licensed to The Apache Software Foundation, http://www.apache.org/ 5 6 Benchmarking 172.16.16.4 (be patient)
此时的CPU使用情况，如下所示：
可以使用命令查看hpa调度的情况，如下所示：
注意：我后来观察了一下下面的信息，发现没有进行Pod的伸缩的报错原因，原来是下面的错误，如下所示：
1 FailedGetMetrics unable to get metrics for resource cpu: failed to get heapster service: the server could not find the requested resource (get services http:heapster:)
具体查看详情信息，如下所示：
1 [root@k8s-master rc]# kubectl describe hpa myweb 2 Name: myweb 3 Namespace: default 4 Labels: <none> 5 Annotations: <none> 6 CreationTimestamp: Sun, 28 Jun 2020 19:47:19 +0800 7 Reference: ReplicationController/myweb 8 Target CPU utilization: 5% 9 Current CPU utilization: <unset> 10 Min replicas: 1 11 Max replicas: 8 12 ReplicationController pods: 2 current / 2 desired 13 Events: 14 FirstSeen LastSeen Count From SubObjectPath Type Reason Message 15 --------- -------- ----- ---- ------------- -------- ------ ------- 16 16m 14m 8 {horizontal-pod-autoscaler } Normal MetricsNotAvailableYet unable to get metrics for resource cpu: failed to get heapster service: the server could not find the requested resource (get services http:heapster:) 17 13m 17s 28 {horizontal-pod-autoscaler } Warning FailedGetMetrics unable to get metrics for resource cpu: failed to get heapster service: the server could not find the requested resource (get services http:heapster:) 18 [root@k8s-master rc]#
我对两个Pod同时压力测试都没有伸缩，没有给我扩容Pod，更别提缩减Pod。
我这里一直测试不出，这里将值调整的更低些，看看效果，如何，先将rc，hpa删除掉哈，然后再创建即可，如下所示：
1 [root@k8s-master ~]# kubectl get all -o wide 2 NAME REFERENCE TARGET CURRENT MINPODS MAXPODS AGE 3 hpa/myweb ReplicationController/myweb 5% <waiting> 1 8 38m 4 5 NAME DESIRED CURRENT READY AGE CONTAINER(S) IMAGE(S) SELECTOR 6 rc/myweb 2 2 2 45m myweb 192.168.110.133:5000/nginx:1.13 app=myweb 7 8 NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR 9 svc/kubernetes 10.254.0.1 <none> 443/TCP 23d <none> 10 11 NAME READY STATUS RESTARTS AGE IP NODE 12 po/myweb-c0rs7 1/1 Running 0 45m 172.16.16.4 k8s-node3 13 po/myweb-jkqc7 1/1 Running 0 45m 172.16.94.3 k8s-node2 14 [root@k8s-master ~]# kubectl get rc -o wide 15 NAME DESIRED CURRENT READY AGE CONTAINER(S) IMAGE(S) SELECTOR 16 myweb 2 2 2 45m myweb 192.168.110.133:5000/nginx:1.13 app=myweb 17 [root@k8s-master ~]# kubectl delete rc myweb 18 replicationcontroller "myweb" deleted 19 [root@k8s-master ~]# kubectl get rc -o wide 20 No resources found. 21 [root@k8s-master ~]# kubectl get all -o wide 22 NAME REFERENCE TARGET CURRENT MINPODS MAXPODS AGE 23 hpa/myweb ReplicationController/myweb 5% <waiting> 1 8 38m 24 25 NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR 26 svc/kubernetes 10.254.0.1 <none> 443/TCP 23d <none> 27 [root@k8s-master ~]# kubectl get hpa -o wide 28 NAME REFERENCE TARGET CURRENT MINPODS MAXPODS AGE 29 myweb ReplicationController/myweb 5% <waiting> 1 8 39m 30 [root@k8s-master ~]# kubectl delete hpa myweb 31 horizontalpodautoscaler "myweb" deleted 32 [root@k8s-master ~]# kubectl get hpa -o wide 33 No resources found. 34 [root@k8s-master ~]# kubectl get all -o wide 35 NAME CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR 36 svc/kubernetes 10.254.0.1 <none> 443/TCP 23d <none> 37 [root@k8s-master ~]#
测不出来，k8s的hpa伸缩和缩减Pod的现象，不管修改内存、CPU参数，还是请求参数，先放到这里吧，以后有机会再来看看。这里没有完成伸缩的具体原因是因为下面这段话。
metrics-server，从 Kubernetes 1.8开始，资源使用指标，例如容器 CPU 和内存使用率，可通过 Metrics API 在 Kubernetes 中获得。此 API 不存储指标值，因此想要获取某个指定节点10分钟前的资源使用量是不可能的。在此之前，kubernetes对容器的监控是通过hepater来完成的。之所以Metrics-server会替换掉hepater成为新的k8s 监控组件，在于其使得kubernetes在监控方面与其他功能保持了一致，不再像是一个割裂开的功能，比如风格统一的监控指标api，kubectl top命令等。这里顺便提一下另外一个组件cadvisor，该监控组件本身并非kubernetes内置，但kubelete内置了部分cadvisor功能，因此kubelete可以获取每个节点的容器监控信息。
5、查看一下我的k8s的版本，如下所示：
1 [root@k8s-master ~]# kubectl version 2 Client Version: version.Info{Major:"1", Minor:"5", GitVersion:"v1.5.2", GitCommit:"269f928217957e7126dc87e6adfa82242bfe5b1e", GitTreeState:"clean", BuildDate:"2017-07-03T15:31:10Z", GoVersion:"go1.7.4", Compiler:"gc", Platform:"linux/amd64"} 3 Server Version: version.Info{Major:"1", Minor:"5", GitVersion:"v1.5.2", GitCommit:"269f928217957e7126dc87e6adfa82242bfe5b1e", GitTreeState:"clean", BuildDate:"2017-07-03T15:31:10Z", GoVersion:"go1.7.4", Compiler:"gc", Platform:"linux/amd64"} 4 [root@k8s-master ~]#
由于heapster停止更新，这里也使用metrics-server来进行监控了。
1 [root@k8s-master ~]# cd k8s/ 2 [root@k8s-master k8s]# ls 3 book-master.war dashboard dashboard.zip deploy health heapster hpa namespace pod rc skydns skydns.zip svc tomcat_demo tomcat_demo.zip 4 [root@k8s-master k8s]# mkdir metrics 5 [root@k8s-master k8s]# cd metrics/ 6 [root@k8s-master metrics]# ls 7 [root@k8s-master metrics]# wget -c https://github.com/kubernetes-sigs/metrics-server/archive/v0.3.6.zip 8 --2020-06-29 10:24:47-- https://github.com/kubernetes-sigs/metrics-server/archive/v0.3.6.zip 9 Resolving github.com (github.com)... 13.250.177.223 10 Connecting to github.com (github.com)|13.250.177.223|:443... connected. 11 HTTP request sent, awaiting response... 302 Found 12 Location: https://codeload.github.com/kubernetes-sigs/metrics-server/zip/v0.3.6 [following] 13 --2020-06-29 10:24:49-- https://codeload.github.com/kubernetes-sigs/metrics-server/zip/v0.3.6 14 Resolving codeload.github.com (codeload.github.com)... 54.251.140.56 15 Connecting to codeload.github.com (codeload.github.com)|54.251.140.56|:443... connected. 16 HTTP request sent, awaiting response... 200 OK 17 Length: unspecified [application/zip] 18 Saving to: ‘v0.3.6.zip’ 19 20 [ <=> ] 9,374,036 183KB/s in 69s 21 22 2020-06-29 10:26:01 (132 KB/s) - ‘v0.3.6.zip’ saved [9374036] 23 24 [root@k8s-master metrics]#
查看下载的zip包并进行解压缩操作，如下所示：
1 [root@k8s-master metrics]# unzip v0.3.6.zip 2 3 [root@k8s-master metrics]# ls 4 metrics-server-0.3.6 v0.3.6.zip 5 [root@k8s-master metrics]# ll 6 total 9156 7 drwxr-xr-x 8 root root 335 Oct 14 2019 metrics-server-0.3.6 8 -rw-r--r-- 1 root root 9374036 Jun 29 10:26 v0.3.6.zip 9 [root@k8s-master metrics]#
查看自己的jdk版本，并进入到指定的目录，如下所示：
1 [root@k8s-master metrics]# ls 2 metrics-server-0.3.6 v0.3.6.zip 3 [root@k8s-master metrics]# cd metrics-server-0.3.6/deploy/1. 4 1.7/ 1.8+/ 5 [root@k8s-master metrics]# cd metrics-server-0.3.6/deploy/1. 6 1.7/ 1.8+/ 7 [root@k8s-master metrics]# cd metrics-server-0.3.6/deploy/1.8+/ 8 [root@k8s-master 1.8+]# ls 9 aggregated-metrics-reader.yaml auth-delegator.yaml auth-reader.yaml metrics-apiservice.yaml metrics-server-deployment.yaml metrics-server-service.yaml resource-reader.yaml 10 [root@k8s-master 1.8+]# java -version 11 openjdk version "1.8.0_181" 12 OpenJDK Runtime Environment (build 1.8.0_181-b13) 13 OpenJDK 64-Bit Server VM (build 25.181-b13, mixed mode) 14 [root@k8s-master 1.8+]#
现在开始修改镜像地址，如下所示：
首先将需要的镜像下载下来，并上传到自己的私有仓库里面，如下所示：
1 [root@k8s-master ~]# docker pull docker.io/htcfive/metrics-server-amd64 2 Using default tag: latest 3 Trying to pull repository docker.io/htcfive/metrics-server-amd64 ... 4 sha256:0122b32b24dcb04ac5131cecdefd8abce0c8a3359605dd17b394acc9fc49de1d: Pulling from docker.io/htcfive/metrics-server-amd64 5 e8d8785a314f: Pull complete 6 b2f4b24bed0d: Pull complete 7 Digest: sha256:0122b32b24dcb04ac5131cecdefd8abce0c8a3359605dd17b394acc9fc49de1d 8 Status: Downloaded newer image for docker.io/htcfive/metrics-server-amd64:latest 9 [root@k8s-master ~]# docker images docker.io/htcfive/metrics-server-amd64 10 REPOSITORY TAG IMAGE ID CREATED SIZE 11 docker.io/htcfive/metrics-server-amd64 latest 77da73af4258 6 months ago 39.9 MB 12 [root@k8s-master ~]# docker tag docker.io/htcfive/metrics-server-amd64:latest 192.168.110.133:5000/docker.io/htcfive/metrics-server-amd64:latest 13 [root@k8s-master ~]# docker push 192.168.110.133:5000/docker.io/htcfive/metrics-server-amd64:latest 14 The push refers to a repository [192.168.110.133:5000/docker.io/htcfive/metrics-server-amd64] 15 7bf3709d22bb: Pushed 16 932da5156413: Pushed 17 latest: digest: sha256:0122b32b24dcb04ac5131cecdefd8abce0c8a3359605dd17b394acc9fc49de1d size: 738 18 [root@k8s-master ~]#
修改镜像地址，metrics-server默认使用node的主机名，但是coredns里面没有物理机主机名的解析，部署的时候添加一个参数--kubelet-preferred-address-types=InternalIP,Hostname,InternalDNS,ExternalDNS,ExternalIP。这里直接通过InternalIP进行访问，忽略客户端证书kubelet-insecure-tls。
1 [root@k8s-master 1.8+]# cat metrics-server-deployment.yaml 2 --- 3 apiVersion: v1 4 kind: ServiceAccount 5 metadata: 6 name: metrics-server 7 namespace: kube-system 8 --- 9 apiVersion: apps/v1 10 kind: Deployment 11 metadata: 12 name: metrics-server 13 namespace: kube-system 14 labels: 15 k8s-app: metrics-server 16 spec: 17 selector: 18 matchLabels: 19 k8s-app: metrics-server 20 template: 21 metadata: 22 name: metrics-server 23 labels: 24 k8s-app: metrics-server 25 spec: 26 serviceAccountName: metrics-server 27 volumes: 28 # mount in tmp so we can safely use from-scratch images and/or read-only containers 29 - name: tmp-dir 30 emptyDir: {} 31 containers: 32 - name: metrics-server 33 # image: k8s.gcr.io/metrics-server-amd64:v0.3.6 34 image: 192.168.110.133:5000/docker.io/htcfive/metrics-server-amd64:latest 35 # imagePullPolicy: Always 36 imagePullPolicy: IfNotPresent 37 command: 38 - /metrics-server 39 - --kubelet-preferred-address-types=InternalIP 40 - --kubelet-insecure-tls 41 volumeMounts: 42 - name: tmp-dir 43 mountPath: /tmp
创建metrics-server报错了，先放置吧，搜了一堆也没有解决，后续再来看看吧。
1 [root@k8s-master 1.8+]# kubectl create -f . 2 Error from server (BadRequest): error when creating "aggregated-metrics-reader.yaml": ClusterRole in version "v1" cannot be handled as a ClusterRole: no kind "ClusterRole" is registered for version "rbac.authorization.k8s.io/v1" 3 Error from server (BadRequest): error when creating "auth-delegator.yaml": ClusterRoleBinding in version "v1beta1" cannot be handled as a ClusterRoleBinding: no kind "ClusterRoleBinding" is registered for version "rbac.authorization.k8s.io/v1beta1" 4 Error from server (BadRequest): error when creating "auth-reader.yaml": RoleBinding in version "v1beta1" cannot be handled as a RoleBinding: no kind "RoleBinding" is registered for version "rbac.authorization.k8s.io/v1beta1" 5 Error from server (AlreadyExists): error when creating "metrics-server-deployment.yaml": serviceaccounts "metrics-server" already exists 6 Error from server (AlreadyExists): error when creating "metrics-server-service.yaml": services "metrics-server" already exists 7 Error from server (BadRequest): error when creating "resource-reader.yaml": ClusterRole in version "v1" cannot be handled as a ClusterRole: no kind "ClusterRole" is registered for version "rbac.authorization.k8s.io/v1" 8 Error from server (BadRequest): error when creating "resource-reader.yaml": ClusterRoleBinding in version "v1" cannot be handled as a ClusterRoleBinding: no kind "ClusterRoleBinding" is registered for version "rbac.authorization.k8s.io/v1" 9 [unable to recognize "metrics-apiservice.yaml": no matches for apiregistration.k8s.io/, Kind=APIService, unable to recognize "metrics-server-deployment.yaml": no matches for apps/, Kind=Deployment] 10 [root@k8s-master 1.8+]#