34 【kubernetes】安装手册

全文参考了两篇中文文档：

1，https://www.cnblogs.com/RainingNight/p/using-kubeadm-to-create-a-cluster.html

2，http://running.iteye.com/blog/2322634

注意：

运行命令是一定要区分是在master节点还是在pods节点上运行的，有些命令只能在master节点执行，有些命令只能在pods节点执行。这个要区分。

运行命令一定要区分清用户是谁，是root还是普通用户。

大步骤：

1，在master节点和pods节点上安装软件；

2，在master节点上启动kubernetes软件，并初始化master节点；

3，在从节点上启动kubernetes软件，并连接到master节点进行注册；

4，通过master启动一个pods，执行一个应用程序（nginx为例）；

5，通过master启动一个服务，将刚才的应用程序关联到这个服务项上；

6，测试master的scale能力，根据服务名瞬间启动一个相同的pods；

1，在master节点和pods节点上安装软件

sudo apt-get update && sudo apt-get install -y apt-transport-https
curl -s http://packages.faasx.com/google/apt/doc/apt-key.gpg | sudo apt-key add -
sudo cat <<EOF >/etc/apt/sources.list.d/kubernetes.list
deb http://mirrors.ustc.edu.cn/kubernetes/apt/ kubernetes-xenial main
EOF
sudo apt-get update
sudo apt-get install -y kubelet kubeadm kubectl

2，在master节点上启动kubernetes软件，并初始化master节点；

2.1，在master节点初始化一个cluster

由于网络原因，我们需要提前拉取k8s初始化需要用到的Images，并添加对应的k8s.gcr.io标签:

## 拉取镜像
docker pull reg.qiniu.com/k8s/kube-apiserver-amd64:v1.10.2
docker pull reg.qiniu.com/k8s/kube-controller-manager-amd64:v1.10.2
docker pull reg.qiniu.com/k8s/kube-scheduler-amd64:v1.10.2
docker pull reg.qiniu.com/k8s/kube-proxy-amd64:v1.10.2
docker pull reg.qiniu.com/k8s/etcd-amd64:3.1.12
docker pull reg.qiniu.com/k8s/pause-amd64:3.1

## 添加Tag
docker tag reg.qiniu.com/k8s/kube-apiserver-amd64:v1.10.2 k8s.gcr.io/kube-apiserver-amd64:v1.10.2
docker tag reg.qiniu.com/k8s/kube-scheduler-amd64:v1.10.2 k8s.gcr.io/kube-scheduler-amd64:v1.10.2
docker tag reg.qiniu.com/k8s/kube-controller-manager-amd64:v1.10.2 k8s.gcr.io/kube-controller-manager-amd64:v1.10.2
docker tag reg.qiniu.com/k8s/kube-proxy-amd64:v1.10.2 k8s.gcr.io/kube-proxy-amd64:v1.10.2
docker tag reg.qiniu.com/k8s/etcd-amd64:3.1.12 k8s.gcr.io/etcd-amd64:3.1.12
docker tag reg.qiniu.com/k8s/pause-amd64:3.1 k8s.gcr.io/pause-amd64:3.1

## 在Kubernetes 1.10 中，增加了CoreDNS，如果使用CoreDNS(默认关闭)，则不需要下面三个镜像。
docker pull reg.qiniu.com/k8s/k8s-dns-sidecar-amd64:1.14.10
docker pull reg.qiniu.com/k8s/k8s-dns-kube-dns-amd64:1.14.10
docker pull reg.qiniu.com/k8s/k8s-dns-dnsmasq-nanny-amd64:1.14.10

docker tag reg.qiniu.com/k8s/k8s-dns-sidecar-amd64:1.14.10 k8s.gcr.io/k8s-dns-sidecar-amd64:1.14.10
docker tag reg.qiniu.com/k8s/k8s-dns-kube-dns-amd64:1.14.10 k8s.gcr.io/k8s-dns-kube-dns-amd64:1.14.10
docker tag reg.qiniu.com/k8s/k8s-dns-dnsmasq-nanny-amd64:1.14.10 k8s.gcr.io/k8s-dns-dnsmasq-nanny-amd64:1.14.10

2.2，初始化cluster

<master-node>: sudo kubeadm init --pod-network-cidr=192.168.0.0/16
注意这里的输出最好能记录在text中，因为后面会用到

2.3，将kubernetes的配置项放到普通用户目录下

mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

这样kubectl会自动寻址到config文件，不用依赖一个严格的root可读的config

2.4，安装各种软件（https://docs.projectcalico.org/v3.3/getting-started/kubernetes/）

2.4.1 安装etcd
kubectl apply -f 
https://docs.projectcalico.org/v3.3/getting-started/kubernetes/installation/hosted/etcd.yaml

2.4.2 安装rbac
kubectl apply -f 
https://docs.projectcalico.org/v3.3/getting-started/kubernetes/installation/rbac.yaml

2.4.3 安装calico
kubectl apply -f 
https://docs.projectcalico.org/v3.3/getting-started/kubernetes/installation/hosted/calico.yaml

2.4.4 确认安装成功
watch kubectl get pods --all-namespaces
结束后Ctrl + C

2.4.5 再次确认
kubectl get nodes -o wide

2.5 在从节点启动kubernetes软件

在从节点上执行：
sudo kubeadm join 192.168.0.8:6443 --token vtyk9m.g4afak37myq3rsdi --discovery-token-ca-cert-hash sha256:19246ce11ba3fc633fe0b21f2f8aaaebd7df9103ae47138dc0dd615f61a32d99 

这里的命令要和2.2的输出保持一致（几个参数可能不一致，按照自己的输出自行修改即可）

如果2.2的输出已经没法找到，可以用以下命令再次得到join语句。

在主节点上执行：
kubeadm token create --print-join-command

然后再在从节点上执行以上得到的join语句

2.6，确认主从节点已经完成启动，需要等几分钟：

主节点上执行
kubectl get nodes

3，创建可用的pod
3.1，创建一个nginx的镜像当做pod内的应用程序

主节点上执行：
kubectl run my-nginx --image=nginx --replicas=1 --port=80 

3.2，确认pod已经生成

主节点上执行：

kubectl get pods

3.3，将该pods发布到kubernetes上，作为一个服务

luwenwei@localhost:~/download/k8s$ kubectl expose deployment my-nginx --port=8080 --target-port=80 
service/my-nginx exposed

kubectl expose deployment <run-deploy-name> --port=<port> --target-port=<target-port> --name=<service-name>
把已经发布的应用<run-deploy-name>暴露成一种服务，原始端口为<port>，对外服务的端口为<target-port>，暴露出来的service名称为<service-name>

　3.4，查看服务是否已经生成

luwenwei@localhost:~$ kubectl get services
NAME         TYPE        CLUSTER-IP    EXTERNAL-IP   PORT(S)    AGE
kubernetes   ClusterIP   10.96.0.1     <none>        443/TCP    108m
my-nginx     ClusterIP   10.98.38.80   <none>        8080/TCP   91m

3.5，访问该服务

luwenwei@localhost:~$ kubectl describe service/my-nginx
Name:              my-nginx
Namespace:         default
Labels:            run=my-nginx
Annotations:       <none>
Selector:          run=my-nginx
Type:              ClusterIP
IP:                10.98.38.80
Port:              <unset>  8080/TCP
TargetPort:        80/TCP
Endpoints:         192.168.244.65:80
Session Affinity:  None
Events:            <none>

获取到IP和port

从节点上执行：
luwenwei@ubuntu:~$ curl 10.98.38.80:8080

<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
         35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

4，扩容该service的pods数
4.1，查看扩容前的pods数

luwenwei@localhost:~$ kubectl get pods
NAME                        READY   STATUS    RESTARTS   AGE
my-nginx-756f645cd7-mg45n   1/1     Running   0          98m

4.2，执行扩容

luwenwei@localhost:~$ kubectl scale deployment my-nginx --replicas=2
deployment.extensions/my-nginx scaled

4.3，查看扩容后的pods信息

luwenwei@localhost:~$ kubectl get pods
NAME                        READY   STATUS              RESTARTS   AGE
my-nginx-756f645cd7-dww7g   0/1     ContainerCreating   0          6s
my-nginx-756f645cd7-mg45n   1/1     Running             0          98m

4.4，查看service的信息

luwenwei@localhost:~$ kubectl describe service/my-nginx
Name:              my-nginx
Namespace:         default
Labels:            run=my-nginx
Annotations:       <none>
Selector:          run=my-nginx
Type:              ClusterIP
IP:                10.98.38.80
Port:              <unset>  8080/TCP
TargetPort:        80/TCP
Endpoints:         192.168.244.65:80,192.168.244.66:80
Session Affinity:  None
Events:            <none>

4.4，反向验证pods和service的对应

luwenwei@localhost:~$ kubectl describe pods | grep IP
IP:                 192.168.244.66
IP:                 192.168.244.65

4.5，访问新的service

luwenwei@ubuntu:~$ curl 10.98.38.80:8080
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
         35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

5，使用完毕后清理cluster信息

5.1，清除node数据

主节点上执行
kubectl drain <node name> --delete-local-data --force --ignore-daemonsets

5.2，删除node节点

主节点上执行
kubectl delete node <node name>

5.3，收回cluster信息

在要移除的节点上，执行：
sudo kubeadm reset

输出结果：

luwenwei@ubuntu:~$ sudo kubeadm reset
[sudo] password for luwenwei: 
[reset] WARNING: changes made to this host by 'kubeadm init' or 'kubeadm join' will be reverted.
[reset] are you sure you want to proceed? [y/N]: y
[preflight] running pre-flight checks
[reset] stopping the kubelet service
[reset] unmounting mounted directories in "/var/lib/kubelet"
[reset] no etcd manifest found in "/etc/kubernetes/manifests/etcd.yaml". Assuming external etcd
[reset] please manually reset etcd to prevent further issues
[reset] deleting contents of stateful directories: [/var/lib/kubelet /etc/cni/net.d /var/lib/dockershim /var/run/kubernetes]
[reset] deleting contents of config directories: [/etc/kubernetes/manifests /etc/kubernetes/pki]
[reset] deleting files: [/etc/kubernetes/admin.conf /etc/kubernetes/kubelet.conf /etc/kubernetes/bootstrap-kubelet.conf /etc/kubernetes/controller-manager.conf /etc/kubernetes/scheduler.conf