k8s Docker 安装
一、运行环境
Centos 7.7
虚拟机内核为 3.10
基础组件版本：
k8s.gcr.io/kube-apiserver:v1.16.0
k8s.gcr.io/kube-controller-manager:v1.16.0
k8s.gcr.io/kube-scheduler:v1.16.0
k8s.gcr.io/kube-proxy:v1.16.0
k8s.gcr.io/pause:3.1
k8s.gcr.io/etcd:3.3.15-0
k8s.gcr.io/coredns:1.6.2
| hostname | ip | resource | role |
|---|---|---|---|
| hsjry-16-114-128 | 172.16.114.128 | 2c2G | master |
| hsjry-16-114-129 | 172.16.114.129 | 2c2G | node1 |
| hsjry-16-114-130 | 172.16.114.130 | 2c2G | node2 |
二、介质准备
需要下载安装 docker 所需要的依赖和 docker 本身。
需要下载上述组件的镜像包
准备 kubeadm kubectl kubelet
cat > /etc/yum.repos.d/kubernetes.repo <<EOF
[kubernetes]
name=Kubernetes Repo
baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
gpgcheck=1
gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
enabled=1
EOF
yum list kubeadm --showduplicates
yum remove kubectl kubeadm kubelet
yum -y install kubectl-1.16.0-0 kubeadm-1.16.0-0 kubelet-1.16.0-0
三、安装 docker
1.残余卸载
安装之前先将原有的
yum remove docker
docker-client
docker-client-latest
docker-common
docker-latest
docker-latest-logrotate
docker-logrotate
docker-selinux
docker-engine-selinux
docker-engine
rm -rf /etc/systemd/system/docker.service.d
rm -rf /var/lib/docker
rm -rf /var/run/docker
2.前期准备
# 关闭防火墙
systemctl stop firewalld && systemctl disable firewalld
iptables -F && iptables -X && iptables -F -t nat && iptables -X -t nat && iptables -P FORWARD ACCEPT
# 关闭 SELinux
setenforce 0
sed -i "s/SELINUX=enforcing/SELINUX=disabled/g" /etc/selinux/config
# 关闭 swapoff
swapoff -a
sed -i '/ swap / s/^(.*)$/#1/g' /etc/fstab
3.rpm 安装 docker
rpm -ivh *.rpm --nodeps --force
4.收尾设置
这里的资源管理方式采用 systemd（可自行根据情况选择）
systemctl enable docker
systemctl start docker
cat << EOF >> /etc/docker/daemon.json
{
"exec-opts": ["native.cgroupdriver=systemd"],
"insecure-registries": ["0.0.0.0/0"]
}
EOF
systemctl restart docker
四、安装 kubernetes
1.前期配置
这个阶段的内容需要在各个节点上执行
base_dir=./k8s
# 加载内核参数
modprobe -- ip_vs
modprobe -- ip_vs_rr
modprobe -- ip_vs_wrr
modprobe -- ip_vs_sh
if [[ $(uname -r |cut -d . -f1) -ge 4 && $(uname -r |cut -d . -f2) -ge 19 ]]; then
modprobe -- nf_conntrack
else
modprobe -- nf_conntrack_ipv4
fi
cat <<EOF > /etc/sysctl.d/k8s.conf
net.bridge.bridge-nf-call-ip6tables = 1
net.bridge.bridge-nf-call-iptables = 1
EOF
# 立即生效
sysctl --system
sysctl -w net.ipv4.ip_forward=1
systemctl stop firewalld && systemctl disable firewalld
swapoff -a || true
setenforce 0 || true
# 这里是将下载好的直接 cp,也可选择 rpm 安装的方式
chmod a+x $base_dir/bin/*
cp $base_dir/bin/* /usr/bin
cp $base_dir/conf/kubelet.service /etc/systemd/system/
mkdir /etc/systemd/system/kubelet.service.d
cp $base_dir/conf/10-kubeadm.conf /etc/systemd/system/kubelet.service.d/
# 获取 docker 的 cgroupDriver
cgroupDriver=$(docker info|grep Cg)
driver=${cgroupDriver##*: }
echo "driver is ${driver}"
mkdir -p /var/lib/kubelet/ || true
# 声明 kubelete 的配置内容
cat <<EOF > /var/lib/kubelet/config.yaml
address: 0.0.0.0
apiVersion: kubelet.config.k8s.io/v1beta1
authentication:
anonymous:
enabled: false
webhook:
cacheTTL: 2m0s
enabled: true
x509:
clientCAFile: /etc/kubernetes/pki/ca.crt
authorization:
mode: Webhook
webhook:
cacheAuthorizedTTL: 5m0s
cacheUnauthorizedTTL: 30s
cgroupDriver: ${driver}
cgroupsPerQOS: true
clusterDNS:
- 10.96.0.10
clusterDomain: cluster.local
configMapAndSecretChangeDetectionStrategy: Watch
containerLogMaxFiles: 5
containerLogMaxSize: 10Mi
contentType: application/vnd.kubernetes.protobuf
cpuCFSQuota: true
cpuCFSQuotaPeriod: 100ms
cpuManagerPolicy: none
cpuManagerReconcilePeriod: 10s
enableControllerAttachDetach: true
enableDebuggingHandlers: true
enforceNodeAllocatable:
- pods
eventBurst: 10
eventRecordQPS: 5
evictionHard:
imagefs.available: 15%
memory.available: 100Mi
nodefs.available: 10%
nodefs.inodesFree: 5%
evictionPressureTransitionPeriod: 5m0s
failSwapOn: true
fileCheckFrequency: 20s
hairpinMode: promiscuous-bridge
healthzBindAddress: 127.0.0.1
healthzPort: 10248
httpCheckFrequency: 20s
imageGCHighThresholdPercent: 85
imageGCLowThresholdPercent: 80
imageMinimumGCAge: 2m0s
iptablesDropBit: 15
iptablesMasqueradeBit: 14
kind: KubeletConfiguration
kubeAPIBurst: 10
kubeAPIQPS: 5
makeIPTablesUtilChains: true
maxOpenFiles: 1000000
maxPods: 110
nodeLeaseDurationSeconds: 40
nodeStatusUpdateFrequency: 10s
oomScoreAdj: -999
podPidsLimit: -1
port: 10250
registryBurst: 10
registryPullQPS: 5
resolvConf: /etc/resolv.conf
rotateCertificates: true
runtimeRequestTimeout: 2m0s
serializeImagePulls: true
staticPodPath: /etc/kubernetes/manifests
streamingConnectionIdleTimeout: 4h0m0s
syncFrequency: 1m0s
volumeStatsAggPeriod: 1m0s
EOF
# 加载镜像包
docker load -i $base_dir/images/images.tar.gz || true
systemctl enable kubelet
2.初始化 master
本次通过 kubeadm 的方式初始化 master 节点
base_dir=./k8s
kubeadm init --config $base_dir/conf/kubeadm.yaml
mkdir ~/.kube
cp /etc/kubernetes/admin.conf ~/.kube/config
kubectl taint nodes --all node-role.kubernetes.io/master-
kubectl apply -f $base_dir/conf/kube-flannel.yaml
sleep 5
kubectl apply -f $base_dir/conf/traefik-config.yaml
这里结束后会输出一个命令，需要 cp 这个命令到 node 节点上敲下，就可以加入master 了
2.初始化 node
# 需执行上述 1 的内容
# 通过 kubeadm create token 创建的 token ，过期时间是24小时，这就是为什么过了一天无法再次使用之前记录的 kube join 原生脚本的原因，也可以运行 kubeadm token create --ttl 0生成一个永不过期的 token，
4.验证
到 master 节点上敲 kubectl get nodes 就可以看到这个集群的信息咯
五、安装 ingress
这里采用 traefix 来作为服务暴露的方式。kubectl apply -f $base_dir/conf/traefik-config.yaml