此案例是以一个主，三个node来部署的，当然node可以根据自己情况部署
192.168.1.130 master 192.168.1.131 node1 192.168.1.132 node2 192.168.1.133 node3
合法的
Enable NTP on master and all nodes :
[root@k-master ~]# yum -y install ntp [root@k-master ~]# systemctl start ntpd [root@k-master ~]# systemctl enable ntpd [root@k-master ~]# hwclock --systohc
[root@k-node1 ~]# yum -y install ntp [root@k-node1 ~]# systemctl start ntpd [root@k-node1 ~]# systemctl enable ntpd [root@k-node1 ~]# hwclock --systohc
[root@k-node2 ~]# yum -y install ntp [root@k-node2 ~]# systemctl start ntpd [root@k-node2 ~]# systemctl enable ntpd [root@k-node2 ~]# hwclock --systohc
[root@k-node3 ~]# yum -y install ntp [root@k-node3 ~]# systemctl start ntpd [root@k-node3 ~]# systemctl enable ntpd [root@k-node3 ~]# hwclock --systohc
Add entries in “/etc/hosts” or reccords in your DNS :
[root@k-master ~]# grep "k-" /etc/hosts
192.168.1.130 k-master
192.168.1.131 k-node1
192.168.1.132 k-node2
192.168.1.133 k-node3
[root@k-node1 ~]# grep "k-" /etc/hosts
192.168.1.130 k-master
192.168.1.131 k-node1
192.168.1.132 k-node2
192.168.1.133 k-node3
[root@k-node2 ~]# grep "k-" /etc/hosts
192.168.1.130 k-master
192.168.1.131 k-node1
192.168.1.132 k-node2
192.168.1.133 k-node3
[root@k-node3 ~]# grep "k-" /etc/hosts
192.168.1.130 k-master
192.168.1.131 k-node1
192.168.1.132 k-node2
192.168.1.133 k-node3
Install required RPM :
- On master :
[root@k-master ~]# yum -y install etcd kubernetes
...
...
...
Installed:
etcd.x86_64 0:2.1.1-2.el7 kubernetes.x86_64 0:1.0.3-0.2.gitb9a88a7.el7
Dependency Installed:
audit-libs-python.x86_64 0:2.4.1-5.el7 checkpolicy.x86_64 0:2.1.12-6.el7
docker.x86_64 0:1.8.2-10.el7.centos docker-selinux.x86_64 0:1.8.2-10.el7.centos
kubernetes-client.x86_64 0:1.0.3-0.2.gitb9a88a7.el7 kubernetes-master.x86_64 0:1.0.3-0.2.gitb9a88a7.el7
kubernetes-node.x86_64 0:1.0.3-0.2.gitb9a88a7.el7 libcgroup.x86_64 0:0.41-8.el7
libsemanage-python.x86_64 0:2.1.10-18.el7 policycoreutils-python.x86_64 0:2.2.5-20.el7
python-IPy.noarch 0:0.75-6.el7 setools-libs.x86_64 0:3.3.7-46.el7
socat.x86_64 0:1.7.2.2-5.el7
Complete!
- On nodes :
[root@k-node1 ~]# yum -y install flannel kubernetes
[root@k-node2 ~]# yum -y install flannel kubernetes
[root@k-node3 ~]# yum -y install flannel kubernetes
Stop the firewall
For for many convenience, we will stop firewalls during this lab :
[root@k-master ~]# systemctl stop firewalld && systemctl disable firewalld
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
Removed symlink /etc/systemd/system/basic.target.wants/firewalld.service.
[root@k-node1 ~]# systemctl stop firewalld && systemctl disable firewalld
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
Removed symlink /etc/systemd/system/basic.target.wants/firewalld.service.
[root@k-node2 ~]# systemctl stop firewalld && systemctl disable firewalld
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
Removed symlink /etc/systemd/system/basic.target.wants/firewalld.service.
[root@k-node3 ~]# systemctl stop firewalld && systemctl disable firewalld
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
Removed symlink /etc/systemd/system/basic.target.wants/firewalld.service.
On Kubernetes master
- Configure “etcd” distributed key-value store :
[root@k-master ~]# egrep -v "^#|^$" /etc/etcd/etcd.conf
ETCD_NAME=default
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_CLIENT_URLS="http://0.0.0.0:2379"
ETCD_ADVERTISE_CLIENT_URLS="http://localhost:2379"
- Kubernetes API server configuration file :
[root@k-master ~]# egrep -v "^#|^$" /etc/kubernetes/apiserver
KUBE_API_ADDRESS="--address=0.0.0.0"
KUBE_API_PORT="--port=8080"
KUBELET_PORT="--kubelet_port=10250"
KUBE_ETCD_SERVERS="--etcd_servers=http://127.0.0.1:2379"
KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=10.254.0.0/16"
KUBE_ADMISSION_CONTROL="--admission_control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ResourceQuota"
KUBE_API_ARGS=""
- Start all Kubernetes services :
[root@k-master ~]# for SERVICE in etcd kube-apiserver kube-controller-manager kube-scheduler
> do
> systemctl restart $SERVICE
> systemctl enable $SERVICE
> done
Created symlink from /etc/systemd/system/multi-user.target.wants/etcd.service to /usr/lib/systemd/system/etcd.service.
Created symlink from /etc/systemd/system/multi-user.target.wants/kube-apiserver.service to /usr/lib/systemd/system/kube-apiserver.service.
Created symlink from /etc/systemd/system/multi-user.target.wants/kube-controller-manager.service to /usr/lib/systemd/system/kube-controller-manager.service.
Created symlink from /etc/systemd/system/multi-user.target.wants/kube-scheduler.service to /usr/lib/systemd/system/kube-scheduler.service
We have now those LISTEN port :
[root@k-master ~]# netstat -ntulp | egrep -v "ntpd|sshd"
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address Foreign Address State PID/Program name
tcp 0 0 127.0.0.1:10251 0.0.0.0:* LISTEN 2913/kube-scheduler
tcp 0 0 127.0.0.1:10252 0.0.0.0:* LISTEN 2887/kube-controlle
tcp 0 0 127.0.0.1:2380 0.0.0.0:* LISTEN 2828/etcd
tcp 0 0 127.0.0.1:7001 0.0.0.0:* LISTEN 2828/etcd
tcp 0 0 127.0.0.1:25 0.0.0.0:* LISTEN 2356/master
tcp6 0 0 :::2379 :::* LISTEN 2828/etcd
tcp6 0 0 :::8080 :::* LISTEN 2858/kube-apiserver
tcp6 0 0 ::1:25 :::* LISTEN 2356/master
- Create “etcd” key :
[root@k-master ~]# etcdctl mk /frederic.wou/network/config '{"Network":"172.17.0.0/16"}'
{"Network":"172.31.0.0/16"}
[root@k-master ~]# etcdctl ls /frederic.wou --recursive
/frederic.wou/network
/frederic.wou/network/config
[root@k-master ~]# etcdctl get /frederic.wou/network/config
{"Network":"172.17.0.0/16"}
On each minion nodes
- flannel configuration:
[root@k-node1 ~]# egrep -v "^#|^$" /etc/sysconfig/flanneld
FLANNEL_ETCD="http://192.168.1.130:2379"
FLANNEL_ETCD_KEY="/frederic.wou/network"
[root@k-node2 ~]# egrep -v "^#|^$" /etc/sysconfig/flanneld
FLANNEL_ETCD="http://192.168.1.130:2379"
FLANNEL_ETCD_KEY="/frederic.wou/network"
[root@k-node3 ~]# egrep -v "^#|^$" /etc/sysconfig/flanneld
FLANNEL_ETCD="http://192.168.1.130:2379"
FLANNEL_ETCD_KEY="/frederic.wou/network"
- Kubernates :
[root@k-node1 ~]# egrep -v "^#|^$" /etc/kubernetes/config
KUBE_LOGTOSTDERR="--logtostderr=true"
KUBE_LOG_LEVEL="--v=0"
KUBE_ALLOW_PRIV="--allow_privileged=false"
KUBE_MASTER="--master=http://192.168.1.130:8080"
[root@k-node2 ~]# egrep -v "^#|^$" /etc/kubernetes/config
KUBE_LOGTOSTDERR="--logtostderr=true"
KUBE_LOG_LEVEL="--v=0"
KUBE_ALLOW_PRIV="--allow_privileged=false"
KUBE_MASTER="--master=http://192.168.1.130:8080"
[root@k-node3 ~]# egrep -v "^#|^$" /etc/kubernetes/config
KUBE_LOGTOSTDERR="--logtostderr=true"
KUBE_LOG_LEVEL="--v=0"
KUBE_ALLOW_PRIV="--allow_privileged=false"
KUBE_MASTER="--master=http://192.168.1.130:8080"
- kubelet :
[root@k-node1 ~]# egrep -v "^#|^$" /etc/kubernetes/kubelet
KUBELET_ADDRESS="--address=0.0.0.0"
KUBELET_PORT="--port=10250"
KUBELET_HOSTNAME="--hostname_override=k-node1"
KUBELET_API_SERVER="--api_servers=http://k-master:8080"
KUBELET_ARGS=""
[root@k-node2 ~]# egrep -v "^#|^$" /etc/kubernetes/kubelet
KUBELET_ADDRESS="--address=0.0.0.0"
KUBELET_PORT="--port=10250"
KUBELET_HOSTNAME="--hostname_override=k-node2"
KUBELET_API_SERVER="--api_servers=http://k-master:8080"
KUBELET_ARGS=""
[root@k-node3 ~]# egrep -v "^#|^$" /etc/kubernetes/kubelet
KUBELET_ADDRESS="--address=0.0.0.0"
KUBELET_PORT="--port=10250"
KUBELET_HOSTNAME="--hostname_override=k-node3"
KUBELET_API_SERVER="--api_servers=http://k-master:8080"
KUBELET_ARGS=""
- Start all services :
[root@k-node1 ~]# for SERVICE in kube-proxy kubelet docker flanneld
> do
> systemctl start $SERVICE
> systemctl enable $SERVICE
> done
Created symlink from /etc/systemd/system/multi-user.target.wants/kube-proxy.service to /usr/lib/systemd/system/kube-proxy.service.
A dependency job for kubelet.service failed. See 'journalctl -xe' for details.
Created symlink from /etc/systemd/system/multi-user.target.wants/kubelet.service to /usr/lib/systemd/system/kubelet.service.
Created symlink from /etc/systemd/system/multi-user.target.wants/docker.service to /usr/lib/systemd/system/docker.service.
Job for flanneld.service failed because a timeout was exceeded. See "systemctl status flanneld.service" and "journalctl -xe" for details.
Created symlink from /etc/systemd/system/multi-user.target.wants/flanneld.service to /usr/lib/systemd/system/flanneld.service.
Created symlink from /etc/systemd/system/docker.service.requires/flanneld.service to /usr/lib/systemd/system/flanneld.service.
[root@k-node2 ~]# for SERVICE in kube-proxy kubelet docker flanneld
> do
> systemctl start $SERVICE
> systemctl enable $SERVICE
> done
Created symlink from /etc/systemd/system/multi-user.target.wants/kube-proxy.service to /usr/lib/systemd/system/kube-proxy.service.
A dependency job for kubelet.service failed. See 'journalctl -xe' for details.
Created symlink from /etc/systemd/system/multi-user.target.wants/kubelet.service to /usr/lib/systemd/system/kubelet.service.
Job for docker.service failed because the control process exited with error code. See "systemctl status docker.service" and "journalctl -xe" for details.
Created symlink from /etc/systemd/system/multi-user.target.wants/docker.service to /usr/lib/systemd/system/docker.service.
Job for flanneld.service failed because a timeout was exceeded. See "systemctl status flanneld.service" and "journalctl -xe" for details.
Created symlink from /etc/systemd/system/multi-user.target.wants/flanneld.service to /usr/lib/systemd/system/flanneld.service.
Created symlink from /etc/systemd/system/docker.service.requires/flanneld.service to /usr/lib/systemd/system/flanneld.service.
[root@k-node3 ~]# for SERVICE in kube-proxy kubelet docker flanneld
> do
> systemctl start $SERVICE
> systemctl enable $SERVICE
> done
Created symlink from /etc/systemd/system/multi-user.target.wants/kube-proxy.service to /usr/lib/systemd/system/kube-proxy.service.
A dependency job for kubelet.service failed. See 'journalctl -xe' for details.
Created symlink from /etc/systemd/system/multi-user.target.wants/kubelet.service to /usr/lib/systemd/system/kubelet.service.
Job for docker.service failed because the control process exited with error code. See "systemctl status docker.service" and "journalctl -xe" for details.
Created symlink from /etc/systemd/system/multi-user.target.wants/docker.service to /usr/lib/systemd/system/docker.service.
Job for flanneld.service failed because a timeout was exceeded. See "systemctl status flanneld.service" and "journalctl -xe" for details.
Created symlink from /etc/systemd/system/multi-user.target.wants/flanneld.service to /usr/lib/systemd/system/flanneld.service.
Created symlink from /etc/systemd/system/docker.service.requires/flanneld.service to /usr/lib/systemd/system/flanneld.service.
Kubernetes is now ready
[root@k-master ~]# kubectl get nodes
NAME LABELS STATUS
192.168.1.131 kubernetes.io/hostname=192.168.1.131 Ready
192.168.1.132 kubernetes.io/hostname=192.168.1.132 Ready
192.168.1.133 kubernetes.io/hostname=192.168.1.133 Ready
Troubleshooting
Unable to start Docker on minion nodes
[root@k-node1 ~]# systemctl start docker
Job for docker.service failed because the control process exited with error code. See "systemctl status docker.service" and "journalctl -xe" for details
Check “ntp” service :
[root@k-node1 ~]# ntpq -p
remote refid st t when poll reach delay offset jitter
==============================================================================
+173.ip-37-59-12 36.224.68.195 2 u - 64 7 32.539 -0.030 0.477
*moz75-1-78-194- 213.251.128.249 2 u 4 64 7 30.108 -0.988 0.967
-ntp.tuxfamily.n 138.96.64.10 2 u 67 64 7 25.934 -1.495 0.504
+x1.f2tec.de 10.2.0.1 2 u 62 64 7 32.307 -0.044 0.466
Is “flanneld” up & running ?
[root@k-node1 ~]# ip addr show dev flannel0
3: flannel0: <POINTOPOINT,MULTICAST,NOARP,UP,LOWER_UP> mtu 1472 qdisc pfifo_fast state UNKNOWN qlen 500
link/none
inet 172.17.85.0/16 scope global flannel0
valid_lft forever preferred_lft forever
Is this node able to connect to “etcd” master :
[root@k-node1 ~]# curl -s -L http://192.168.1.130:2379/version
{"etcdserver":"2.1.1","etcdcluster":"2.1.0"}[root@k-node1 ~]
Is “kube-proxy” service running ?
[root@k-node1 ~]# systemctl status kube-proxy
● kube-proxy.service - Kubernetes Kube-Proxy Server
Loaded: loaded (/usr/lib/systemd/system/kube-proxy.service; disabled; vendor preset: disabled)
Active: active (running) since Wed 2016-02-03 14:50:25 CET; 1min 0s ago
Docs: https://github.com/GoogleCloudPlatform/kubernetes
Main PID: 2072 (kube-proxy)
CGroup: /system.slice/kube-proxy.service
└─2072 /usr/bin/kube-proxy --logtostderr=true --v=0 --master=http://192.168.1.130:8080
Feb 03 14:50:25 k-node1 systemd[1]: Started Kubernetes Kube-Proxy Server.
Feb 03 14:50:25 k-node1 systemd[1]: Starting Kubernetes Kube-Proxy Server...
Try to manually start Docker daemon :
[root@k-node1 ~]# cat /run/flannel/docker
DOCKER_OPT_BIP="--bip=172.17.85.1/24"
DOCKER_OPT_IPMASQ="--ip-masq=true"
DOCKER_OPT_MTU="--mtu=1472"
DOCKER_NETWORK_OPTIONS=" --bip=172.17.85.1/24 --ip-masq=true --mtu=1472 "
[root@k-node1 ~]# /usr/bin/docker daemon -D --selinux-enabled --bip=172.17.85.1/24 --ip-masq=true --mtu=1472
...
...
...
INFO[0001] Docker daemon commit=a01dc02/1.8.2 execdriver=native-0.2 graphdriver=devicemapper version=1.8.2-el7.centos
#转自http://frederic-wou.net/kubernetes-first-step-on-centos-7-2/