kubernetes 1.6 集群实践 （四）

Flannel 概述

k8s集群pod的ip在不同的网段中，只依赖主机网络无法正常访问，需要使用flannel网络组件，Flannel 能让在所有节点 (Master、Node) 上的 Pod 、sverice 互联互通。

参考资料：http://dockone.io/article/618

安装flannel

全部节点都要安装

$ yum install -y flannel
$ rpm -qa|grep flannel
flannel-0.7.1-2.el7.x86_64

编辑配置文件

修改systemd 启动文件

cat >  /usr/lib/systemd/system/flanneld.service  << EOF
[Unit]
Description=Flanneld overlay address etcd agent
After=network.target
After=network-online.target
Wants=network-online.target
After=etcd.service
Before=docker.service

[Service]
Type=notify
EnvironmentFile=/etc/sysconfig/flanneld
EnvironmentFile=-/etc/sysconfig/docker-network
ExecStart=/usr/bin/flanneld-start $FLANNEL_OPTIONS
ExecStartPost=/usr/libexec/flannel/mk-docker-opts.sh -k DOCKER_NETWORK_OPTIONS -d /run/flannel/docker
Restart=on-failure

[Install]
WantedBy=multi-user.target
RequiredBy=docker.service
EOF

修改/etc/sysconfig/flanneld 配置文件

cat > /etc/sysconfig/flanneld<< EOF
# Flanneld configuration options  

# etcd url location.  Point this to the server where etcd runs
FLANNEL_ETCD_ENDPOINTS="https://192.168.19.101:2379,https://192.168.19.102:2379,https://192.168.19.103:2379"

# etcd config key.  This is the configuration key that flannel queries
# For address range assignment
FLANNEL_ETCD_PREFIX="/kube-centos/network"

# Any additional options that you want to pass
FLANNEL_OPTIONS="-etcd-cafile=/etc/kubernetes/ssl/ca.pem -etcd-certfile=/etc/kubernetes/ssl/kubernetes.pem -etcd-keyfile=/etc/kubernetes/ssl/kubernetes-key.pem"
EOF

FLANNEL_ETCD_PREFIX 是flannel在etcd中的配置路径

在etcd中添加flannel的配置信息

etcdctl --endpoints=https://192.168.19.101:2379 --ca-file=/etc/kubernetes/ssl/ca.pem --cert-file=/etc/kubernetes/ssl/kubernetes.pem  --key-file=/etc/kubernetes/ssl/kubernetes-key.pem set /kube-centos/network/config '{"Network":"10.250.0.0/16", "SubnetLen": 24, "Backend": {"Type": "host-gw"}}'

Network 将是pod分配的网段

在全部节点启动flannel

$ systemctl daemon-reload && systemctl start flanneld && systemctl enable flanneld

在etcd上检查flannel各节点信息

查看已经分配的网段

$  etcdctl --endpoints=https://192.168.19.101:2379 --ca-file=/etc/kubernetes/ssl/ca.pem --cert-file=/etc/kubernetes/ssl/kubernetes.pem  --key-file=/etc/kubernetes/ssl/kubernetes-key.pem ls /kube-centos/network/subnets
/kube-centos/network/subnets/10.250.64.0-24
/kube-centos/network/subnets/10.250.6.0-24
/kube-centos/network/subnets/10.250.94.0-24
/kube-centos/network/subnets/10.250.60.0-24

查看网段的详细信息

$ etcdctl --endpoints=https://192.168.19.101:2379 --ca-file=/etc/kubernetes/ssl/ca.pem --cert-file=/etc/kubernetes/ssl/kubernetes.pem  --key-file=/etc/kubernetes/ssl/kubernetes-key.pem get /kube-centos/network/subnets/10.250.6.0-24
{"PublicIP":"192.168.19.102","BackendType":"vxlan","BackendData":{"VtepMAC":"c6:a3:86:77:76:0b"}}

docker

安装docker

$ yum install docker -y

配置docker

$ cat >/usr/lib/systemd/system/docker.service <<EOF
[Unit]
Description=Docker Application Container Engine
Documentation=http://docs.docker.com
After=network.target
Wants=docker-storage-setup.service
Requires=docker-cleanup.timer

[Service]
Type=notify
NotifyAccess=all
EnvironmentFile=-/run/containers/registries.conf
EnvironmentFile=-/etc/sysconfig/docker
EnvironmentFile=-/etc/sysconfig/docker-storage
EnvironmentFile=-/etc/sysconfig/docker-network
EnvironmentFile=-/run/flannel/docker
Environment=GOTRACEBACK=crash
Environment=DOCKER_HTTP_HOST_COMPAT=1
Environment=PATH=/usr/libexec/docker:/usr/bin:/usr/sbin
ExecStart=/usr/bin/dockerd-current 
          --add-runtime docker-runc=/usr/libexec/docker/docker-runc-current 
          --default-runtime=docker-runc 
          --exec-opt native.cgroupdriver=systemd 
          --userland-proxy-path=/usr/libexec/docker/docker-proxy-current 
          --graph=/opt/docker  
          --storage-opt=dm.loopmetadatasize=10G    --storage-opt=dm.loopdatasize=400G   
          --registry-mirror=https://*******e.mirror.aliyuncs.com　
          --insecure-registry 192.168.7.0/24  
          $OPTIONS 
          $DOCKER_STORAGE_OPTIONS 
          $DOCKER_NETWORK_OPTIONS 
          $ADD_REGISTRY 
          $BLOCK_REGISTRY 
          $INSECURE_REGISTRY
          $REGISTRIES
ExecReload=/bin/kill -s HUP $MAINPID
LimitNOFILE=1048576
LimitNPROC=1048576
LimitCORE=infinity
TimeoutStartSec=0
Restart=on-abnormal
MountFlags=slave
KillMode=process

[Install]
WantedBy=multi-user.target
EOF

启动docker

$ systemctl daemon-reload && systemctl start docker

如果有docker0网桥删除方法

$ ifconfig docker0 down
$ brctl delbr docker0