k8s v1.9.9 二进制部署 （6）所有节点安装flannel网络插件和docker

1.二进制包安装flannel

$cd /root/

$wget https://github.com/coreos/flannel/releases/download/v0.10.0/flannel-v0.10.0-linux-amd64.tar.gz

$tar zxvf flannel-v0.10.0-linux-amd64.tar.gz

$ll

$mkdir /usr/libexec/flannel/

$cp mk-docker-opts.sh /usr/libexec/flannel/

$cp flanneld /usr/bin

2.配置和启动fannel

2.1 配置文件 /usr/lib/systemd/system/flanneld.service

 

注：

这一步建议vim编辑，我用eof方式写入时，发现vim将变量内容当成变量读取，结果为空，导致丢失部分内容。

$vim /usr/lib/systemd/system/flanneld.service

[Unit]

Description=Flanneld overlay address etcd agent

After=network.target

After=network-online.target

Wants=network-online.target

After=etcd.service

Before=docker.service

[Service]

Type=notify

EnvironmentFile=/etc/sysconfig/flanneld

EnvironmentFile=-/etc/sysconfig/docker-network

ExecStart=/usr/bin/flanneld  

 -etcd-endpoints=${FLANNEL_ETCD_ENDPOINTS}

 -etcd-prefix=${FLANNEL_ETCD_PREFIX}

 $FLANNEL_OPTIONS

ExecStartPost=/usr/libexec/flannel/mk-docker-opts.sh -k DOCKER_NETWORK_OPTIONS -d /run/flannel/docker

Restart=on-failure

[Install]

WantedBy=multi-user.target

RequiredBy=docker.service

2.2 配置文件 /etc/sysconfig/flanneld

注：红色内容写入etcd集群IP地址。

$cat > /etc/sysconfig/flanneld <<eof

# Flanneld configuration options  

# etcd url location.  Point this to the server where etcd runs

FLANNEL_ETCD_ENDPOINTS="https://192.168.7.131:2379,https://192.168.7.132:2379,https://192.168.7.133:2379"

# etcd config key.  This is the configuration key that flannel queries

# For address range assignment

FLANNEL_ETCD_PREFIX="/kube-centos/network"

# Any additional options that you want to pass

FLANNEL_OPTIONS="-etcd-cafile=/etc/kubernetes/ssl/ca.pem -etcd-certfile=/etc/kubernetes/ssl/kubernetes.pem -etcd-keyfile=/etc/kubernetes/ssl/kubernetes-key.pem"

eof

2.3 在etcd中创建网络配置（仅在master配置）

注：红色内容写入etcd集群IP地址。

$etcdctl --endpoints=https://192.168.7.131:2379,https://192.168.7.132:2379,https://192.168.7.133:2379

 --ca-file=/etc/kubernetes/ssl/ca.pem

 --cert-file=/etc/kubernetes/ssl/kubernetes.pem

 --key-file=/etc/kubernetes/ssl/kubernetes-key.pem

 mkdir /kube-centos/network

 

$etcdctl --endpoints=https://192.168.7.131:2379,https://192.168.7.132:2379,https://192.168.7.133:2379  

 --ca-file=/etc/kubernetes/ssl/ca.pem

 --cert-file=/etc/kubernetes/ssl/kubernetes.pem

 --key-file=/etc/kubernetes/ssl/kubernetes-key.pem

 mk /kube-centos/network/config '{"Network":"172.18.0.0/16","SubnetLen":24,"Backend":{"Type":"vxlan"}}'

 

3.启动flannel

systemctl daemon-reload

systemctl enable flanneld

systemctl start flanneld

systemctl status flanneld

 

4.验证

注: 这时三个节点上查看ip，都能看到生成了一块flannel网卡，IP都是172.18.0.0/16地址段的。

$ip a

任意节点运行

$etcdctl --endpoints=${ETCD_ENDPOINTS}

  --ca-file=/etc/kubernetes/ssl/ca.pem

  --cert-file=/etc/kubernetes/ssl/kubernetes.pem

  --key-file=/etc/kubernetes/ssl/kubernetes-key.pem

  ls /kube-centos/network/subnets

 

$etcdctl --endpoints=${ETCD_ENDPOINTS}

  --ca-file=/etc/kubernetes/ssl/ca.pem

  --cert-file=/etc/kubernetes/ssl/kubernetes.pem

  --key-file=/etc/kubernetes/ssl/kubernetes-key.pem

  get /kube-centos/network/config

 

5.安装docker

5.1.上传docker文件夹到三个节点上

注：这里离线部署的，可以yum部署建议版本18或者19开头的。

(未测试)二进制部署docker地址如下，下载解压后，将docker二进制执行文件放到/usr/bin/下，继续后面的配置，应该没问题的。

https://download.docker.com/linux/static/stable/x86_64/docker-19.03.1.tgz

 

$cd /root/

$上传docker文件夹，用winscp工具上传的。

$tar zcvf docker.tar.gz docker/

$scp docker.tar.gz root@192.168.7.132:/root

$scp docker.tar.gz root@192.168.7.133:/root

5.2.安装

#/etc/yum.repos.d/CentOS-Base.repo 需要有，部分离线包需要更新。

$cd /root/docker/

$yum -y localinstall lvm2/*.rpm container-selinux/*.rpm device-mapper-persistent-data/*.rpm

$yum -y localinstall docker-ce-cli/*.rpm

5.3.查看版本

docker --version

6. 配置文件 /usr/lib/systemd/system/docker.service

$cd /root/

$vim /usr/lib/systemd/system/docker.service

[Unit]

Description=Docker Application Container Engine

Documentation=https://docs.docker.com

After=network-online.target firewalld.service

Wants=network-online.target

[Service]

Type=notify

EnvironmentFile=-/run/flannel/docker

ExecStart=/usr/bin/dockerd  --exec-opt native.cgroupdriver=systemd

                            $OPTIONS

                            $DOCKER_STORAGE_OPTIONS

                            $DOCKER_NETWORK_OPTIONS

                            $ADD_REGISTRY

                            $BLOCK_REGISTRY

                            $REGISTRIES

ExecReload=/bin/kill -s HUP $MAINPID

LimitNOFILE=infinity

LimitNPROC=infinity

LimitCORE=infinity

TimeoutStartSec=0

Delegate=yes

KillMode=process

Restart=on-failure

StartLimitBurst=3

StartLimitInterval=60s

[Install]

WantedBy=multi-user.target

7.启动docker：

systemctl daemon-reload

systemctl enable docker

systemctl start docker

systemctl status docker

 

8.验证docker

所有节点都生成了一块docker0网卡，且地址与flannel同网段。