zoukankan      html  css  js  c++  java
  • 【Kubernetes学习之二】Kubernetes集群安装

    环境
      centos 7

    Kubernetes有三种安装方式:yum、二进制、kubeadm,这里演示kubeadm。

    一、准备工作
    1、软件版本

    软件 版本
    kubernetes v1.15.3
    CentOS7.6 CentOS Linux release 7.6.1810(Core)
    Docker docker-ce-19.03.1-3.el7.x86_64
    flannel 0.11.0


    2、集群拓扑

    IP 角色 主机名
    192.168.118.106 master node106 k8s-master
    192.168.118.107 node01 node107 k8s-node01
    192.168.118.108 node02 node108 k8s-node02

    节点及网络规划如下:

    3、系统设置
    3.1 配置主机名-/etc/hosts

    192.168.118.106    node106 k8s-master
    192.168.118.107    node107 k8s-node01
    192.168.118.108    node108 k8s-node02

    3.2 关闭防火墙

    [root@node106 ~]# yum install -y net-tools
    #关闭防火墙
    [root@node106 ~]# systemctl stop firewalld
    #禁用防火墙
    [root@node106 ~]# systemctl disable firewalld

    3.3 文件权限相关-关闭SELinux
    目的是允许容器访问主机文件系统。

    [root@node106 ~]# sed -i 's#SELINUX=enforcing#SELINUX=disabled#g' /etc/selinux/config
    [root@node106 ~]# setenforce 0

    3.4 关闭swap
    kubernetes的想法是将实例紧密包装到尽可能接近100%,所有的部署应该与CPU/内存限制固定在一起,所以如果调度程序发送一个pod到一台机器,它不应该使用交换。
    设计者不想交换,因为它会减慢速度,所以关闭swap主要是为了性能考虑。当然为了一些节省资源的场景,比如运行容器数量较多,可添加kubelet参数 --fail-swap-on=false来解决

    [root@node106 ~]# swapoff -a
    [root@node106 ~]# sed -i 's/.*swap.*/#&/' /etc/fstab

    3.5 配置转发参数
    RHEL/CentOS7上由于iptables被绕过而导致流量路由不正确的问题,需要将net.bridge.bridge-nf-call-iptables在sysctl配置中设置为1。
    确保br_netfilter在此步骤之前加载了模块。这可以通过运行来完成lsmod | grep br_netfilter。要加载它显式调用modprobe br_netfilter。
    (1)首先查看是否加载了模块br_netfilter

    [root@node106 ~]# lsmod | grep br_netfilter
    br_netfilter           22256  0 
    bridge                151336  1 br_netfilter

    (2)如果未加载,进行加载

    [root@node106 ~]# modprobe br_netfilter

    (3)配置net.bridge.bridge-nf-call-iptables

    [root@node106 ~]# cat <<EOF >  /etc/sysctl.d/k8s.conf
    > net.bridge.bridge-nf-call-ip6tables = 1
    > net.bridge.bridge-nf-call-iptables = 1
    > EOF
    [root@node106 ~]# sysctl --system
    * Applying /usr/lib/sysctl.d/00-system.conf ...
    net.bridge.bridge-nf-call-ip6tables = 0
    net.bridge.bridge-nf-call-iptables = 0
    net.bridge.bridge-nf-call-arptables = 0
    * Applying /usr/lib/sysctl.d/10-default-yama-scope.conf ...
    kernel.yama.ptrace_scope = 0
    * Applying /usr/lib/sysctl.d/50-default.conf ...
    kernel.sysrq = 16
    kernel.core_uses_pid = 1
    net.ipv4.conf.default.rp_filter = 1
    net.ipv4.conf.all.rp_filter = 1
    net.ipv4.conf.default.accept_source_route = 0
    net.ipv4.conf.all.accept_source_route = 0
    net.ipv4.conf.default.promote_secondaries = 1
    net.ipv4.conf.all.promote_secondaries = 1
    fs.protected_hardlinks = 1
    fs.protected_symlinks = 1
    * Applying /etc/sysctl.d/99-sysctl.conf ...
    * Applying /etc/sysctl.d/k8s.conf ...
    net.bridge.bridge-nf-call-ip6tables = 1
    net.bridge.bridge-nf-call-iptables = 1
    * Applying /etc/sysctl.conf ...

    4、docker安装
    (1)设置docker源。

    [root@node106 ~]# yum install -y yum-utils device-mapper-persistent-data lvm2
    [root@node106 ~]# yum-config-manager --add-repo http://mirrors.aliyun.com/docker-ce/linux/centos/docker-ce.repo

    #禁用docker-ce-edge开发版本 不稳定

    [root@node106 ~]# yum-config-manager --disable docker-ce-edge
    [root@node106 ~]# yum makecache fast

    (2)查看目前官方仓库的docker版本

    [root@node106 yum.repos.d]# yum list docker-ce.x86_64  --showduplicates |sort -r
     * updates: mirrors.aliyun.com
    Loading mirror speeds from cached hostfile
    Loaded plugins: fastestmirror
     * extras: mirrors.aliyun.com
    docker-ce.x86_64            3:19.03.1-3.el7                     docker-ce-stable
    docker-ce.x86_64            3:19.03.0-3.el7                     docker-ce-stable
    docker-ce.x86_64            3:18.09.8-3.el7                     docker-ce-stable
    docker-ce.x86_64            3:18.09.7-3.el7                     docker-ce-stable
    docker-ce.x86_64            3:18.09.6-3.el7                     docker-ce-stable
    docker-ce.x86_64            3:18.09.5-3.el7                     docker-ce-stable
    docker-ce.x86_64            3:18.09.4-3.el7                     docker-ce-stable
    docker-ce.x86_64            3:18.09.3-3.el7                     docker-ce-stable
    docker-ce.x86_64            3:18.09.2-3.el7                     docker-ce-stable
    docker-ce.x86_64            3:18.09.1-3.el7                     docker-ce-stable
    docker-ce.x86_64            3:18.09.0-3.el7                     docker-ce-stable
    docker-ce.x86_64            18.06.3.ce-3.el7                    docker-ce-stable
    docker-ce.x86_64            18.06.2.ce-3.el7                    docker-ce-stable
    docker-ce.x86_64            18.06.1.ce-3.el7                    docker-ce-stable
    docker-ce.x86_64            18.06.0.ce-3.el7                    docker-ce-stable
    docker-ce.x86_64            18.03.1.ce-1.el7.centos             docker-ce-stable
    docker-ce.x86_64            18.03.0.ce-1.el7.centos             docker-ce-stable
    docker-ce.x86_64            17.12.1.ce-1.el7.centos             docker-ce-stable
    docker-ce.x86_64            17.12.0.ce-1.el7.centos             docker-ce-stable
    docker-ce.x86_64            17.09.1.ce-1.el7.centos             docker-ce-stable
    docker-ce.x86_64            17.09.0.ce-1.el7.centos             docker-ce-stable
    docker-ce.x86_64            17.06.2.ce-1.el7.centos             docker-ce-stable
    docker-ce.x86_64            17.06.1.ce-1.el7.centos             docker-ce-stable
    docker-ce.x86_64            17.06.0.ce-1.el7.centos             docker-ce-stable
    docker-ce.x86_64            17.03.3.ce-1.el7                    docker-ce-stable
    docker-ce.x86_64            17.03.2.ce-1.el7.centos             docker-ce-stable
    docker-ce.x86_64            17.03.1.ce-1.el7.centos             docker-ce-stable
    docker-ce.x86_64            17.03.0.ce-1.el7.centos             docker-ce-stable
     * base: mirrors.aliyun.com
    Available Packages

    (3)安装docker

    [root@node106 ~]# yum install docker-ce-19.03.1-3.el7 -y

    (4)配置国内镜像仓库加速器

    [root@node106 ~]# mkdir -p /etc/docker
    [root@node106 ~]# tee /etc/docker/daemon.json <<-'EOF'
    {
      "registry-mirrors": ["https://qr09dqf9.mirror.aliyuncs.com"]
    }
    EOF

    (5)启动docker

    [root@node106 ~]# systemctl daemon-reload
    [root@node106 ~]# systemctl enable docker
    [root@node106 ~]# systemctl start docker

    验证:

    [root@node106 ~]# docker -v
    Docker version 19.03.1, build 74b1e89

    5、安装kubernetes相关组件
    5.1设置国内kubernetes阿里云源。

    cat <<EOF > /etc/yum.repos.d/kubernetes.repo
    [kubernetes]
    name=Kubernetes
    baseurl=https://mirrors.aliyun.com/kubernetes/yum/repos/kubernetes-el7-x86_64/
    enabled=1
    gpgcheck=1
    repo_gpgcheck=1
    gpgkey=https://mirrors.aliyun.com/kubernetes/yum/doc/yum-key.gpg https://mirrors.aliyun.com/kubernetes/yum/doc/rpm-package-key.gpg
    EOF

    #增量更新缓存

    [root@node106 ~]# yum makecache fast -y

    #查看kubectl kubelet kubeadm列表

    [root@node106 ~]# yum list kubectl kubelet kubeadm
    Loaded plugins: fastestmirror
    Loading mirror speeds from cached hostfile
     * base: mirrors.aliyun.com
     * extras: mirrors.aliyun.com
     * updates: mirrors.aliyun.com
    Available Packages
    kubeadm.x86_64                                                                         1.15.3-0                                                                   kubernetes
    kubectl.x86_64                                                                         1.15.3-0                                                                   kubernetes
    kubelet.x86_64                                                                         1.15.3-0 

    #安装

    [root@node106 ~]# yum install -y kubectl kubelet kubeadm

    开启kubelet服务

    [root@node106 ~]# systemctl enable --now kubelet
    Created symlink from /etc/systemd/system/multi-user.target.wants/kubelet.service to /usr/lib/systemd/system/kubelet.service.

     6、加载IPVS内核

    ipvs (IP Virtual Server) 实现了传输层负载均衡,也就是我们常说的4层LAN交换,作为 Linux 内核的一部分。ipvs运行在主机上,在真实服务器集群前充当负载均衡器。ipvs可以将基于TCP和UDP的服务请求转发到真实服务器上,并使真实服务器的服务在单个 IP 地址上显示为虚拟服务。pod的负载均衡是用kube-proxy来实现的,实现方式有两种,一种是默认的iptables,一种是ipvs,ipvs比iptable的性能更好而已。
    (1)加载ipvs内核,使node节点kube-proxy支持ipvs代理规则。

    #检查有没有开启
    [root@node106 ~]# cut -f1 -d " "  /proc/modules | grep -e ip_vs -e nf_conntrack_ipv4
    ip_vs_sh
    ip_vs_wrr
    ip_vs_rr
    ip_vs
    nf_conntrack_ipv4
    
    #如果没有开启 使用如下命令开启:
    modprobe ip_vs
    modprobe ip_vs_rr
    modprobe ip_vs_wrr
    modprobe ip_vs_sh
    modprobe nf_conntrack_ipv4

    (2)添加到开机启动文件/etc/rc.local里面

    cat <<EOF >> /etc/rc.local
    
    modprobe ip_vs
    modprobe ip_vs_rr modprobe ip_vs_wrr modprobe ip_vs_sh
    modprobe nf_conntrack_ipv4
    EOF

    (3)ipvs还需要ipset

    [root@node106 ~]# yum install ipset ipvsadm -y

    参考:

    k8s集群中ipvs负载详解
    如何在kubernetes中启用ipvs

    kubernetes的ipvs模式和iptables模式

    二、安装master节点
    1、初始化master节点
    kubeadm init --kubernetes-version=v1.15.3

    1)初始化遇到的问题
    第一次init:

    [root@node106 ~]# kubeadm init --kubernetes-version=v1.15.3
    [init] Using Kubernetes version: v1.15.3
    [preflight] Running pre-flight checks
        [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
        [WARNING SystemVerification]: this Docker version is not on the list of validated versions: 19.03.1. Latest validated version: 18.09
        [WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
    error execution phase preflight: [preflight] Some fatal errors occurred:
        [ERROR NumCPU]: the number of available CPUs 1 is less than the required 2
    [preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`

    分析:
    警告1:[WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd".
    警告2:[WARNING SystemVerification]: this Docker version is not on the list of validated versions: 19.03.1. Latest validated version: 18.09
    版本警告
    警告3:[WARNING Service-Kubelet]: kubelet service is not enabled, please run 'systemctl enable kubelet.service'
    解决:[root@node106 ~]# systemctl enable kubelet.service
    错误1:[ERROR NumCPU]:设置虚拟机CPU核心数>1个即可

    第二次init:

    [root@node106 ~]# kubeadm init --kubernetes-version=v1.15.3
    [init] Using Kubernetes version: v1.15.3
    [preflight] Running pre-flight checks
        [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
        [WARNING SystemVerification]: this Docker version is not on the list of validated versions: 19.03.1. Latest validated version: 18.09
    [preflight] Pulling images required for setting up a Kubernetes cluster
    [preflight] This might take a minute or two, depending on the speed of your internet connection
    [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
    error execution phase preflight: [preflight] Some fatal errors occurred:
        [ERROR ImagePull]: failed to pull image k8s.gcr.io/kube-apiserver:v1.15.3: output: Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
    , error: exit status 1
        [ERROR ImagePull]: failed to pull image k8s.gcr.io/kube-controller-manager:v1.15.3: output: Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
    , error: exit status 1
        [ERROR ImagePull]: failed to pull image k8s.gcr.io/kube-scheduler:v1.15.3: output: Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
    , error: exit status 1
        [ERROR ImagePull]: failed to pull image k8s.gcr.io/kube-proxy:v1.15.3: output: Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
    , error: exit status 1
        [ERROR ImagePull]: failed to pull image k8s.gcr.io/pause:3.1: output: Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
    , error: exit status 1
        [ERROR ImagePull]: failed to pull image k8s.gcr.io/etcd:3.3.10: output: Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
    , error: exit status 1
        [ERROR ImagePull]: failed to pull image k8s.gcr.io/coredns:1.3.1: output: Error response from daemon: Get https://k8s.gcr.io/v2/: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
    , error: exit status 1
    [preflight] If you know what you are doing, you can make a check non-fatal with `--ignore-preflight-errors=...`
    [root@node106 ~]# 

    分析:
    错误1:[ERROR ImagePull] 拉取Image失败,因为连接的是google服务器,可以根据报错中版本号使用docker拉取或者通过kubeadm config images list查看需要下载的版本

    [root@node106 ~]# kubeadm config images list
    W0906 11:12:52.841583   16407 version.go:98] could not fetch a Kubernetes version from the internet: unable to get URL "https://dl.k8s.io/release/stable-1.txt": Get https://dl.k8s.io/release/stable-1.txt: net/http: request canceled while waiting for connection (Client.Timeout exceeded while awaiting headers)
    W0906 11:12:52.841780   16407 version.go:99] falling back to the local client version: v1.15.3
    k8s.gcr.io/kube-apiserver:v1.15.3
    k8s.gcr.io/kube-controller-manager:v1.15.3
    k8s.gcr.io/kube-scheduler:v1.15.3
    k8s.gcr.io/kube-proxy:v1.15.3
    k8s.gcr.io/pause:3.1
    k8s.gcr.io/etcd:3.3.10
    k8s.gcr.io/coredns:1.3.1
    [root@node106 ~]# 

    (2)准备镜像
    mirrorgooglecontainers 在 docker hub 同步了所有 k8s 最新的镜像,先从这儿下载,然后修改 tag 即可。
    #拉镜像

    [root@node106 ~]# kubeadm config images list |sed -e 's/^/docker pull /g' -e 's#k8s.gcr.io#mirrorgooglecontainers#g' |sh -x && docker pull coredns/coredns:1.3.1

    #修改tag,将镜像标记为k8s.gcr.io的名称

    [root@node106 ~]# docker images |grep mirrorgooglecontainers |awk '{print "docker tag ",$1":"$2,$1":"$2}' |sed -e 's#mirrorgooglecontainers#k8s.gcr.io#2' |sh -x && docker tag coredns/coredns:1.3.1 k8s.gcr.io/coredns:1.3.1

    #删除无用的镜像

    [root@node106 ~]# docker images | grep mirrorgooglecontainers | awk '{print "docker rmi " $1":"$2}' | sh -x && docker rmi coredns/coredns:1.3.1

    最终:

    [root@node106 ~]# docker images
    REPOSITORY                           TAG                 IMAGE ID            CREATED             SIZE
    k8s.gcr.io/kube-proxy                v1.15.3             232b5c793146        2 weeks ago         82.4MB
    k8s.gcr.io/kube-apiserver            v1.15.3             5eb2d3fc7a44        2 weeks ago         207MB
    k8s.gcr.io/kube-controller-manager   v1.15.3             e77c31de5547        2 weeks ago         159MB
    k8s.gcr.io/kube-scheduler            v1.15.3             703f9c69a5d5        2 weeks ago         81.1MB
    k8s.gcr.io/coredns                   1.3.1               eb516548c180        7 months ago        40.3MB
    k8s.gcr.io/etcd                      3.3.10              2c4adeb21b4f        9 months ago        258MB
    k8s.gcr.io/pause                     3.1                 da86e6ba6ca1        20 months ago       742kB
    [root@node106 ~]# 

      (3)初始化 

    因为后面要安装网络插件flannel ,所有这里要添加参数, --pod-network-cidr=10.244.0.0/16,10.244.0.0/16是flannel插件固定使用的ip段,它的值取决于你准备安装哪个网络插件

    [root@node106 ~]# kubeadm init --pod-network-cidr=10.244.0.0/16 --kubernetes-version=1.15.3
    [init] Using Kubernetes version: v1.15.3
    [preflight] Running pre-flight checks
        [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
        [WARNING SystemVerification]: this Docker version is not on the list of validated versions: 19.03.1. Latest validated version: 18.09
    [preflight] Pulling images required for setting up a Kubernetes cluster
    [preflight] This might take a minute or two, depending on the speed of your internet connection
    [preflight] You can also perform this action in beforehand using 'kubeadm config images pull'
    [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
    [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
    [kubelet-start] Activating the kubelet service
    [certs] Using certificateDir folder "/etc/kubernetes/pki"
    [certs] Generating "etcd/ca" certificate and key
    [certs] Generating "etcd/server" certificate and key
    [certs] etcd/server serving cert is signed for DNS names [node106 localhost] and IPs [192.168.118.106 127.0.0.1 ::1]
    [certs] Generating "apiserver-etcd-client" certificate and key
    [certs] Generating "etcd/peer" certificate and key
    [certs] etcd/peer serving cert is signed for DNS names [node106 localhost] and IPs [192.168.118.106 127.0.0.1 ::1]
    [certs] Generating "etcd/healthcheck-client" certificate and key
    [certs] Generating "ca" certificate and key
    [certs] Generating "apiserver" certificate and key
    [certs] apiserver serving cert is signed for DNS names [node106 kubernetes kubernetes.default kubernetes.default.svc kubernetes.default.svc.cluster.local] and IPs [10.96.0.1 192.168.118.106]
    [certs] Generating "apiserver-kubelet-client" certificate and key
    [certs] Generating "front-proxy-ca" certificate and key
    [certs] Generating "front-proxy-client" certificate and key
    [certs] Generating "sa" key and public key
    [kubeconfig] Using kubeconfig folder "/etc/kubernetes"
    [kubeconfig] Writing "admin.conf" kubeconfig file
    [kubeconfig] Writing "kubelet.conf" kubeconfig file
    [kubeconfig] Writing "controller-manager.conf" kubeconfig file
    [kubeconfig] Writing "scheduler.conf" kubeconfig file
    [control-plane] Using manifest folder "/etc/kubernetes/manifests"
    [control-plane] Creating static Pod manifest for "kube-apiserver"
    [control-plane] Creating static Pod manifest for "kube-controller-manager"
    [control-plane] Creating static Pod manifest for "kube-scheduler"
    [etcd] Creating static Pod manifest for local etcd in "/etc/kubernetes/manifests"
    [wait-control-plane] Waiting for the kubelet to boot up the control plane as static Pods from directory "/etc/kubernetes/manifests". This can take up to 4m0s
    [apiclient] All control plane components are healthy after 20.007081 seconds
    [upload-config] Storing the configuration used in ConfigMap "kubeadm-config" in the "kube-system" Namespace
    [kubelet] Creating a ConfigMap "kubelet-config-1.15" in namespace kube-system with the configuration for the kubelets in the cluster
    [upload-certs] Skipping phase. Please see --upload-certs
    [mark-control-plane] Marking the node node106 as control-plane by adding the label "node-role.kubernetes.io/master=''"
    [mark-control-plane] Marking the node node106 as control-plane by adding the taints [node-role.kubernetes.io/master:NoSchedule]
    [bootstrap-token] Using token: unqj7v.wr7yvcj8i7wan93g
    [bootstrap-token] Configuring bootstrap tokens, cluster-info ConfigMap, RBAC Roles
    [bootstrap-token] configured RBAC rules to allow Node Bootstrap tokens to post CSRs in order for nodes to get long term certificate credentials
    [bootstrap-token] configured RBAC rules to allow the csrapprover controller automatically approve CSRs from a Node Bootstrap Token
    [bootstrap-token] configured RBAC rules to allow certificate rotation for all node client certificates in the cluster
    [bootstrap-token] Creating the "cluster-info" ConfigMap in the "kube-public" namespace
    [addons] Applied essential addon: CoreDNS
    [addons] Applied essential addon: kube-proxy
    
    Your Kubernetes control-plane has initialized successfully!
    
    To start using your cluster, you need to run the following as a regular user:
    
      mkdir -p $HOME/.kube
      sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
      sudo chown $(id -u):$(id -g) $HOME/.kube/config
    
    You should now deploy a pod network to the cluster.
    Run "kubectl apply -f [podnetwork].yaml" with one of the options listed at:
      https://kubernetes.io/docs/concepts/cluster-administration/addons/
    
    Then you can join any number of worker nodes by running the following on each as root:
    
    kubeadm join 192.168.118.106:6443 --token unqj7v.wr7yvcj8i7wan93g 
        --discovery-token-ca-cert-hash sha256:011f55be71445e7031ac7a582afc7a4350cdf6d8ae8bef790d2517634d93f337

     后续操作:

    [root@node106 ~]# mkdir -p $HOME/.kube
    [root@node106 ~]# sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
    [root@node106 ~]# sudo chown $(id -u):$(id -g) $HOME/.kube/config

    kubectl命令默认从$HOME/.kube/config这个位置读取配置,不做这个操作,使用kubectl会报错。

     2、给pod配置网络

    Flannel是 CoreOS 团队针对 Kubernetes 设计的一个覆盖网络(Overlay Network)工具,其目的在于帮助每一个使用 Kuberentes 的 CoreOS 主机拥有一个完整的子网。
    Flannel通过给每台宿主机分配一个子网的方式为容器提供虚拟网络,它基于Linux TUN/TAP,使用UDP封装IP包来创建overlay网络,并借助etcd维护网络的分配情况。

    #下载Flannel插件配置

    [root@node106 ~]# wget https://raw.githubusercontent.com/coreos/flannel/master/Documentation/kube-flannel.yml
    [root@node106 ~]# ll
    total 20
    -rw-------. 1 root root  1779 Aug 15 14:39 anaconda-ks.cfg
    -rw-r--r--  1 root root 12487 Sep  6 16:42 kube-flannel.yml

    #kube安装kube-flannel.yml

    [root@node106 ~]# kubectl apply -f kube-flannel.yml
    podsecuritypolicy.policy/psp.flannel.unprivileged created
    clusterrole.rbac.authorization.k8s.io/flannel created
    clusterrolebinding.rbac.authorization.k8s.io/flannel created
    serviceaccount/flannel created
    configmap/kube-flannel-cfg created
    daemonset.apps/kube-flannel-ds-amd64 created
    daemonset.apps/kube-flannel-ds-arm64 created
    daemonset.apps/kube-flannel-ds-arm created
    daemonset.apps/kube-flannel-ds-ppc64le created
    daemonset.apps/kube-flannel-ds-s390x created

    #查看Master状态  

    [root@node106 ~]# kubectl get pods --all-namespaces
    NAMESPACE     NAME                              READY   STATUS    RESTARTS   AGE
    kube-system   coredns-5c98db65d4-dwjfs          1/1     Running   0          3h57m
    kube-system   coredns-5c98db65d4-xxdr2          1/1     Running   0          3h57m
    kube-system   etcd-node106                      1/1     Running   0          3h56m
    kube-system   kube-apiserver-node106            1/1     Running   0          3h56m
    kube-system   kube-controller-manager-node106   1/1     Running   0          3h56m
    kube-system   kube-flannel-ds-amd64-srdxz       1/1     Running   0          2m32s
    kube-system   kube-proxy-8mxmm                  1/1     Running   0          3h57m
    kube-system   kube-scheduler-node106            1/1     Running   0          3h56m

    不是running状态,就说明出错了,通过以下操作来来排错:
    查看描述:

    [root@node106 ~]# kubectl describe pod kube-scheduler-node106 -n kube-system

    查看日志:

    [root@node106 ~]# kubectl logs kube-scheduler-node106 -n kube-system

    参考:Flannel安装部署 

    三、安装node节点

    1、下载需要的镜像
    node107和node108节点只需要安装kube-proxy和pause镜像

    [root@node107 ~]# docker images
    REPOSITORY              TAG                 IMAGE ID            CREATED             SIZE
    k8s.gcr.io/kube-proxy   v1.15.3             232b5c793146        2 weeks ago         82.4MB
    k8s.gcr.io/pause        3.1                 da86e6ba6ca1        20 months ago       742kB

    2、添加节点
    在master上初始化节点成功时,最后有一个kubeadm join,就是用来添加节点的
    在node107和node108上操作:

    [root@node107 ~]# kubeadm join 192.168.118.106:6443 --token unqj7v.wr7yvcj8i7wan93g 
    >     --discovery-token-ca-cert-hash sha256:011f55be71445e7031ac7a582afc7a4350cdf6d8ae8bef790d2517634d93f337
    [preflight] Running pre-flight checks
        [WARNING IsDockerSystemdCheck]: detected "cgroupfs" as the Docker cgroup driver. The recommended driver is "systemd". Please follow the guide at https://kubernetes.io/docs/setup/cri/
        [WARNING SystemVerification]: this Docker version is not on the list of validated versions: 19.03.1. Latest validated version: 18.09
    [preflight] Reading configuration from the cluster...
    [preflight] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -oyaml'
    [kubelet-start] Downloading configuration for the kubelet from the "kubelet-config-1.15" ConfigMap in the kube-system namespace
    [kubelet-start] Writing kubelet configuration to file "/var/lib/kubelet/config.yaml"
    [kubelet-start] Writing kubelet environment file with flags to file "/var/lib/kubelet/kubeadm-flags.env"
    [kubelet-start] Activating the kubelet service
    [kubelet-start] Waiting for the kubelet to perform the TLS Bootstrap...
    
    This node has joined the cluster:
    * Certificate signing request was sent to apiserver and a response was received.
    * The Kubelet was informed of the new secure connection details.
    
    Run 'kubectl get nodes' on the control-plane to see this node join the cluster.

    提示:如果执行join命令时提示token过期,按照提示在Master 上执行kubeadm token create生成一个新的token。如果忘记token,可以使用kubeadm token list查看。

    四、验证集群
    1、节点状态

    [root@node106 ~]# kubectl get nodes
    NAME      STATUS   ROLES    AGE     VERSION
    node106   Ready    master   4h53m   v1.15.3
    node107   Ready    <none>   101s    v1.15.3
    node108   Ready    <none>   82s     v1.15.3

    2、组件状态

    [root@node106 ~]# kubectl get cs
    NAME                 STATUS    MESSAGE             ERROR
    controller-manager   Healthy   ok                  
    scheduler            Healthy   ok                  
    etcd-0               Healthy   {"health":"true"}   

    3、服务账户

    [root@node106 ~]# kubectl get serviceaccount
    NAME      SECRETS   AGE
    default   1         5h1m

    4、集群信息

    [root@node106 ~]# kubectl cluster-info
    Kubernetes master is running at https://192.168.118.106:6443
    KubeDNS is running at https://192.168.118.106:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
    
    To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

    5、验证dns功能

    [root@node106 ~]# kubectl run curl --image=radial/busyboxplus:curl -it
    kubectl run --generator=deployment/apps.v1 is DEPRECATED and will be removed in a future version. Use kubectl run --generator=run-pod/v1 or kubectl create instead.
    If you don't see a command prompt, try pressing enter.
    [ root@curl-6bf6db5c4f-dn65h:/ ]$ nslookup kubernetes.default
    Server:    10.96.0.10
    Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local
    
    Name:      kubernetes.default
    Address 1: 10.96.0.1 kubernetes.default.svc.cluster.local

    五、案例验证
    创建一个nginx的service试一下集群是否可用。

    (1)创建并运行deployment

    [root@node106 ~]# kubectl run nginx --replicas=2 --labels="run=load-balancer-example" --image=nginx  --port=80
    kubectl run --generator=deployment/apps.v1 is DEPRECATED and will be removed in a future version. Use kubectl run --generator=run-pod/v1 or kubectl create instead.
    deployment.apps/nginx created

    (2)把服务通过nodeport的形式暴露出来

    [root@node106 ~]# kubectl expose deployment nginx --type=NodePort --name=example-service
    service/example-service exposed
    #查看服务的详细信息
    [root@node106 ~]# kubectl expose deployment nginx --type=NodePort --name=example-service
    service/example-service exposed
    [root@node106 ~]# kubectl describe service example-service
    Name:                     example-service
    Namespace:                default
    Labels:                   run=load-balancer-example
    Annotations:              <none>
    Selector:                 run=load-balancer-example
    Type:                     NodePort
    IP:                       10.108.73.249
    Port:                     <unset>  80/TCP
    TargetPort:               80/TCP
    NodePort:                 <unset>  32168/TCP
    Endpoints:                10.244.1.4:80,10.244.2.2:80
    Session Affinity:         None
    External Traffic Policy:  Cluster
    Events:                   <none>
    
    #查看服务状态
    [root@node106 ~]# kubectl get service
    NAME              TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)        AGE
    example-service   NodePort    10.108.73.249   <none>        80:32168/TCP   91s
    kubernetes        ClusterIP   10.96.0.1       <none>        443/TCP        44h
    [root@node106 ~]# 
    
    #查看pod
    应用的配置和当前状态信息保存在 etcd 中,执行 kubectl get pod 时 API Server 会从 etcd 中读取这些数据。
    [root@node106 ~]# kubectl get pods
    NAME                     READY   STATUS    RESTARTS   AGE
    curl-6bf6db5c4f-dn65h    1/1     Running   2          39h
    nginx-5c47ff5dd6-hjxq8   1/1     Running   0          3m10s
    nginx-5c47ff5dd6-qj9k2   1/1     Running   0          3m10s

    (3)访问服务IP

    [root@node106 ~]# curl 10.108.73.249:80
    <!DOCTYPE html>
    <html>
    <head>
    <title>Welcome to nginx!</title>
    <style>
        body {
             35em;
            margin: 0 auto;
            font-family: Tahoma, Verdana, Arial, sans-serif;
        }
    </style>
    </head>
    <body>
    <h1>Welcome to nginx!</h1>
    <p>If you see this page, the nginx web server is successfully installed and
    working. Further configuration is required.</p>
    
    <p>For online documentation and support please refer to
    <a href="http://nginx.org/">nginx.org</a>.<br/>
    Commercial support is available at
    <a href="http://nginx.com/">nginx.com</a>.</p>
    
    <p><em>Thank you for using nginx.</em></p>
    </body>
    </html>

    访问endpoint,与访问服务ip结果相同。这些IP只能在 Kubernetes Cluster中的容器和节点访问。endpoint与service 之间有映射关系。service实际上是负载均衡着后端的endpoint。其原理是通过iptables实现的

    [root@node106 ~]# curl 10.244.1.4:80
    [root@node106 ~]# curl 10.244.2.2:80

    访问节点ip,与访问集群ip相同,可以在集群外部访问

    [root@node106 ~]# curl 192.168.118.107:32168
    [root@node106 ~]# curl 192.168.118.108:32168

    整个部署过程是这样的:
    ① kubectl 发送部署请求到 API Server。
    ② API Server 通知 Controller Manager 创建一个 deployment 资源。
    ③ Scheduler 执行调度任务,将两个副本 Pod 分发到 node01 和 node02。
    ④ node01 和 node02 上的kubelet 在各自的节点上创建并运行 Pod。
    flannel 会为每个 Pod 都分配 IP。

    参考:
    yum安装Kubernetes
    二进制安装Kubernetes
    kubeadm安装Kubernetes
    手把手教你在CentOS上搭建Kubernetes集群
    官网Installing kubeadm

    Kubernetes最新版本安装过程和注意事项

    kubeadm安装kubernetes1.13集群

  • 相关阅读:
    BZOJ1588:[HNOI2002]营业额统计——题解
    POJ2079:Triangle——题解
    POJ3384:Feng Shui——题解
    POJ3525:Most Distant Point from the Sea——题解
    POJ1279:Art Gallery——题解
    重大改革!Python,最接近人工智能的语言~将被加入高考科目!
    滴滴招聘-各类测试人才
    DJI大疆创新招聘-自动化测试工程师
    自动化测试ROI实践
    自动化测试ROI实践
  • 原文地址:https://www.cnblogs.com/cac2020/p/11433289.html
Copyright © 2011-2022 走看看