【k8s】kubeadm快速部署Kubernetes

1.Kubernetes 架构图###

kubeadm是官方社区推出的一个用于快速部署kubernetes集群的工具。 这个工具能通过两条指令完成一个kubernetes集群的部署:

# 创建一个 Master 节点 
$ kubeadm init

# 将一个 Node 节点加入到当前集群中
$ kubeadm join <Master节点的IP和端口 >

使用它构建集群时，大致可分为如下几步:

• 在Master及各Node安装Docker、kubelet及kubeadm，并以系统守护进程的方式启动Docker和kubelet服务。
• 在Master节点上通过kubeadminit命令进行集群初始化。
• 各Node通过kubeadmjoin命令加入初始化完成的集群中。
• 在集群上部署网络附件，如flannel或Calico等以提供Service网络及Pod网络。

2.Kubernetes 环境准备###

1⃣️关闭防火墙:

systemctl stop firewalld && systemctl disable firewalld

2⃣️关闭selinux:

sed -i 's/enforcing/disabled/' /etc/selinux/config 
setenforce 0

3⃣️关闭swap:

#临时
swapoff -a  
#永久关闭swap分区
sudo sed -ri 's/.*swap.*/#&/' /etc/fstab

4⃣️添加主机名与IP对应关系(记得设置主机名):

cat /etc/hosts
10.211.55.3 k8s-master 
10.211.55.4 k8s-node1
10.211.55.6 k8s-node2
10.211.55.7 k8s-node3

5⃣️将桥接的IPv4流量传递到iptables的链:

cat > /etc/sysctl.d/k8s.conf << EOF net.bridge.bridge-nf-call-ip6tables = 1 net.bridge.bridge-nf-call-iptables = 1 EOF
sysctl --system

3.所有节点安装Docker/kubeadm/kubelet###

Kubernetes默认CRI(容器运行时)为Docker，因此先安装Docker。
1⃣️安装Docker：

sudo su
wget https://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo -O /etc/yum.repos.d/docker-ce.repo
yum -y install docker-ce-18.06.1.ce-3.el7
systemctl enable docker && systemctl start docker
docker --version

2⃣️添加阿里云YUM软件源：

[root@k8s-master yum.repos.d]# cat /etc/yum.repos.d/kubernetes.repo 
[kubernetes]
name=Kubernetes 
baseurl=http://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64 
enabled=1
gpgcheck=0
repo_gpgcheck=0 
gpgkey=http://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg 
http://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg

3⃣️安装kubeadm，kubelet和kubectl

yum install -y kubelet kubeadm kubectl
systemctl enable kubelet && systemctl start kubelet

4.部署Kubernetes Master###

master节点初始化集群
1⃣️由于默认拉取镜像地址k8s.gcr.io国内无法访问，这里指定阿里云镜像仓库地址。

kubeadm init --apiserver-advertise-address=10.211.55.3 --image-repository registry.aliyuncs.com/google_containers --service-cidr=10.1.0.0/16 --pod-network-cidr=10.244.0.0/16

执行init后：

Your Kubernetes control-plane has initialized successfully!

To start using your cluster, you need to run the following as a regular user:

  mkdir -p $HOME/.kube
  sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
  sudo chown $(id -u):$(id -g) $HOME/.kube/config

You should now deploy a pod network to the cluster.
Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
  https://kubernetes.io/docs/concepts/cluster-administration/addons/

Then you can join any number of worker nodes by running the following on each as root:

kubeadm join 10.211.55.3:6443 --token eehcsb.r8tnvj7ov436x63q 
    --discovery-token-ca-cert-hash sha256:583f343b5d55ff96c5a83ccc82444de3c0313adf135d8a980507932aa2f51e1d 

如果出错，执行：

[root@k8s-master yum.repos.d]# kubeadm reset

2⃣️使用kubectl工具:

[root@k8s-master yum.repos.d]# mkdir -p $HOME/.kube
[root@k8s-master yum.repos.d]# sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
[root@k8s-master yum.repos.d]# sudo chown $(id -u):$(id -g) $HOME/.kube/config
[root@k8s-master yum.repos.d]# kubectl get nodes
NAME         STATUS     ROLES    AGE     VERSION
k8s-master   NotReady   master   8m33s   v1.17.0

另外， 提示我们还需要创建网络，并且让其他节点执行kubeadm join...加入集群。

5.创建网络###

如果不创建网络，查看pod状态时，可以看到kube-dns组件是阻塞状态，集群是不可用的：

[root@k8s-master yum.repos.d]# kubectl get pod -n kube-system
NAME                                 READY   STATUS    RESTARTS   AGE
coredns-9d85f5447-n9kkd              0/1     Pending   0          25m
coredns-9d85f5447-s9ms2              0/1     Pending   0          25m
etcd-k8s-master                      1/1     Running   0          25m
kube-apiserver-k8s-master            1/1     Running   0          25m
kube-controller-manager-k8s-master   1/1     Running   0          25m
kube-proxy-94wtc                     1/1     Running   0          5m41s
kube-proxy-h8q8h                     1/1     Running   0          5m45s
kube-proxy-kxmxt                     1/1     Running   0          25m
kube-proxy-nxtpq                     1/1     Running   0          5m44s
kube-scheduler-k8s-master            1/1     Running   0          25m

master节点部署网络插件fannel
基于kubeadm部署时，flannel同样运行为Kubernetes集群的附件，以Pod的形式部署运行于每个集群节点上以接受Kubernetes集群管理。
事实上，也可以直接将flannel程序包安装并以守护进程的方式运行于集群节点上，即以非托管的方式运行。部署方式既可以是获取其资源配置清单于本地而后部署于集群中，也可以直接在线进行应用部署。
部署命令是“kubectlapply”或“kubectlcreate”，例如，下面的命令将直接使用在线的配置清单进行flannel部署：

kubectl apply -f https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml

参考 https://github.com/coreos/flannel
至此master节点就完成了！

[root@k8s-master parallels]# kubectl get cs
NAME                 STATUS    MESSAGE             ERROR
controller-manager   Healthy   ok                  
scheduler            Healthy   ok                  
etcd-0               Healthy   {"health":"true"}   

6.加入Kubernetes Node###

Master各组件运行正常后即可将各Node添加至集群中。
向集群添加新节点，执行在kubeadm init输出的kubeadm join命令:

[root@k8s-node1 yum.repos.d]# kubeadm join 10.211.55.3:6443 --token eehcsb.r8tnvj7ov436x63q 
>     --discovery-token-ca-cert-hash sha256:583f343b5d55ff96c5a83ccc82444de3c0313adf135d8a980507932aa2f51e1d 
...
This node has joined the cluster:
* Certificate signing request was sent to apiserver and a response was received.
* The Kubelet was informed of the new secure connection details.

Run 'kubectl get nodes' on the control-plane to see this node join the cluster.

查看kubernetes集群状态：

[root@k8s-master parallels]# kubectl get nodes
NAME         STATUS   ROLES    AGE   VERSION
k8s-master   Ready    master   17h   v1.17.0
k8s-node1    Ready    <none>   17h   v1.17.0
k8s-node2    Ready    <none>   17h   v1.17.0
k8s-node3    Ready    <none>   17h   v1.17.0

查看k8s部署过程中用到的镜像：
master节点

[root@k8s-master parallels]# docker images
REPOSITORY                                                        TAG                 IMAGE ID            CREATED             SIZE
registry.aliyuncs.com/google_containers/kube-proxy                v1.17.0             7d54289267dc        4 days ago          116MB
registry.aliyuncs.com/google_containers/kube-controller-manager   v1.17.0             5eb3b7486872        4 days ago          161MB
registry.aliyuncs.com/google_containers/kube-scheduler            v1.17.0             78c190f736b1        4 days ago          94.4MB
registry.aliyuncs.com/google_containers/kube-apiserver            v1.17.0             0cae8d5cc64c        4 days ago          171MB
registry.aliyuncs.com/google_containers/coredns                   1.6.5               70f311871ae1        5 weeks ago         41.6MB
registry.aliyuncs.com/google_containers/etcd                      3.4.3-0             303ce5db0e90        6 weeks ago         288MB
quay.io/coreos/flannel                                            v0.11.0-amd64       ff281650a721        10 months ago       52.6MB
registry.aliyuncs.com/google_containers/pause                     3.1                 da86e6ba6ca1        24 months ago       742kB

Node节点

[root@k8s-node1 parallels]# docker images
REPOSITORY                                           TAG                 IMAGE ID            CREATED             SIZE
registry.aliyuncs.com/google_containers/kube-proxy   v1.17.0             7d54289267dc        3 days ago          116MB
quay.io/coreos/flannel                               v0.11.0-amd64       ff281650a721        10 months ago       52.6MB
registry.aliyuncs.com/google_containers/pause        3.1                 da86e6ba6ca1        24 months ago       742kB