10分钟入门kubernetes(上)

kubernetes简称k8s, 主要用途是automate deployment, scaling, and managment of containerized applications。是目前非常火的docker集群部署管理方案，适用于部署microservice架构。是google15年的经验积淀。这就要说说google为啥要用container了，因为那个时候还没有virtual machine这个技术，当然现在发现这个方向的应用前景比virtual machine集群效果要好。提到kubernetes，不能不说CNCF，CNCF出品，必属精品。现在k8s和envoy(lyft的神作)打得火热，感觉是未来云服务商的主流架构，GKE是google k8s engine也是kubernetes的原始应用，不过Azure好像现在也开始支持k8s，amazon的ecs如果不是用的k8s做后端那么也势必会迁移到k8s。

alternative solution to k8s:

• docker swarm
• hashicorp nomad
• apache mesos
• rancher

k8s的诞生

k8s的前身是google的Borg，google没有选择开源Borg，估计是因为内部依赖比较多，而是从头build了k8s，所以说k8s的架构从一开始就站在google15年大型container管理经验的肩上，根正苗红。Google 2015年发表了著名的Borg paper “Large-scale cluster management at Google with Borg”，从而将秘密昭示天下。

下面是一个趣闻，说mesos是怎么诞生的，当时伯克利的一群科学家和google做Borg的工程师酒后闲聊，弄明白了Borg的核心原理，然后就搞出了mesos，所以mesos反而是抢在了k8s的前面，提前成为了很多大公司包括twitter，verizon在内的解决方案，k8s这个正品反而由硅谷的startup率先尝试。所以江湖传言说需求大还是要靠mesos就是这么来的，但是k8s正在飞速的证明自己的成熟度。

k8s 架构

k8s could be deployed as services on VMs or bare-metal machines.

k8s 的安装，本文仅限于单机版minikube

安装minikube

https://github.com/kubernetes/minikube

安装kubectl

https://kubernetes.io/docs/tasks/tools/install-kubectl/#install-kubectl-binary-via-curl

安装virtualbox

https://www.virtualbox.org/wiki/Downloads

 k8s dashboard

minikube dashboard

在dashboard里create app redis，name redis，然后查看：

$ kubectl get pods
NAME                     READY     STATUS    RESTARTS   AGE
redis-76d7657885-rk6tm   1/1       Running   0          1m
$ kubectl get deployment
NAME      DESIRED   CURRENT   UP-TO-DATE   AVAILABLE   AGE
redis     1         1         1            1           2m
$ kubectl get replicaset
NAME               DESIRED   CURRENT   READY     AGE
redis-76d7657885   1         1         1         2m
$ kubectl get nodes
NAME       STATUS    ROLES     AGE       VERSION
minikube   Ready     <none>    14d       v1.9.0

kubectl 应用

check redis logs并与redis进行简单交互：

$ kubectl logs redis-76d7657885-rk6tm
1:C 27 Feb 21:47:54.141 # oO0OoO0OoO0Oo Redis is starting oO0OoO0OoO0Oo
1:C 27 Feb 21:47:54.141 # Redis version=4.0.8, bits=64, commit=00000000, modified=0, pid=1, just started
1:C 27 Feb 21:47:54.141 # Warning: no config file specified, using the default config. In order to specify a config file use redis-server /path/to/redis.conf
1:M 27 Feb 21:47:54.146 * Running mode=standalone, port=6379.
1:M 27 Feb 21:47:54.147 # WARNING: The TCP backlog setting of 511 cannot be enforced because /proc/sys/net/core/somaxconn is set to the lower value of 128.
1:M 27 Feb 21:47:54.147 # Server initialized
1:M 27 Feb 21:47:54.147 * Ready to accept connections
$ kubectl exec -ti redis-76d7657885-rk6tm -- redis-cli
127.0.0.1:6379> set foo bar
OK
127.0.0.1:6379> get foo
"bar"

查看pod的yaml输出，object存于etcd中

$get pods redis-76d7657885-rk6tm -o yaml
apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: 2018-02-27T21:47:43Z
  generateName: redis-76d7657885-
  labels:
    k8s-app: redis
    pod-template-hash: "3283213441"
  name: redis-76d7657885-rk6tm
  ...

刚才是通过dashboard的create button创建的pod，下面通过yaml创建pod：

$ cat pod.yaml 
apiVersion: v1
kind: Pod
metadata:
  name: qingge
spec:
  containers:
  - name: nginx
    image: nginx
$ kubectl create -f pod.yaml
pod "qingge" created

显示pods

$ kubectl get pods
NAME                     READY     STATUS              RESTARTS   AGE
qingge                  0/1       ContainerCreating   0          6s
redis-76d7657885-rk6tm   1/1       Running             0          16m
$ kubectl get pods
NAME                     READY     STATUS    RESTARTS   AGE
qingge                  1/1       Running   0          1m
redis-76d7657885-rk6tm   1/1       Running   0          17m

启动proxy：

$ kubectl proxy
Starting to serve on 127.0.0.1:8001

访问proxy：

$ curl localhost:8001
{
  "paths": [
    "/api",
    "/api/v1",
    "/apis",
    "/apis/",
...

  $ curl localhost:8001/api/v1

  {

  "kind": "APIResourceList",

  "groupVersion": "v1",

  "resources": [

    {

      "name": "bindings",

      "singularName": "",

      "namespaced": true,

      "kind": "Binding",

      "verbs": [

        "create"

      ]

... 

delete pods:

$ kubectl delete pods qingge

　pod "qingge" deleted

$ kubectl get pods

NAME                     READY     STATUS    RESTARTS   AGE

redis-76d7657885-rk6tm   1/1       Running   0          28m

namespace:

$ kubectl get ns
NAME          STATUS    AGE
default       Active    14d
kube-public   Active    14d
kube-system   Active    14d
$ kubectl get pods --all-namespaces
NAMESPACE     NAME                                    READY     STATUS    RESTARTS   AGE
default       redis-76d7657885-rk6tm                  1/1       Running   0          33m
kube-system   kube-addon-manager-minikube             1/1       Running   1          14d
kube-system   kube-dns-54cccfbdf8-mhr9h               3/3       Running   3          14d
kube-system   kubernetes-dashboard-77d8b98585-5r7qv   1/1       Running   1          14d
kube-system   storage-provisioner                     1/1       Running   1          14d

添加quota 限制：

$ kubectl create quota qingge --hard=pods=1
resourcequota "qingge" created
$ kubectl get quota
NAME      AGE
qingge   12s
$ kubectl get quota qingge -o yaml
apiVersion: v1
kind: ResourceQuota
metadata:
  creationTimestamp: 2018-02-27T22:24:08Z
  name: qingge
  namespace: default
  resourceVersion: "2185"
  selfLink: /api/v1/namespaces/default/resourcequotas/qingge
  uid: ed5ec545-1c0c-11e8-aa17-08002725a271
spec:
  hard:
    pods: "1"
status:
  hard:
    pods: "1"
  used:
    pods: "1"

$ kubectl create -f pod.yaml

Error from server (Forbidden): error when creating "pod.yaml": pods "qingge" is forbidden: exceeded quota: qingge, requested: pods=1, used: pods=1, limited: pods=1 

修改quota 把spec中的pods改为2，然后就可以添加pod了

$ kubectl edit resourcequota oreilly
resourcequota "qingge" edited
$ kubectl create -f pod.yaml
pod "qingge" created

观察删除redis后会自动被replica恢复：

$ kubectl delete pods redis-76d7657885-rk6tm
pod "redis-76d7657885-rk6tm" deleted
$ kubectl get pods
NAME                     READY     STATUS              RESTARTS   AGE
redis-76d7657885-lnmps   0/1       ContainerCreating   0          2s
redis-76d7657885-rk6tm   0/1       Terminating         0          43m
$ kubectl get pods
NAME                     READY     STATUS        RESTARTS   AGE
redis-76d7657885-lnmps   1/1       Running       0          6s
redis-76d7657885-rk6tm   0/1       Terminating   0          44m
$ kubectl get pods
NAME                     READY     STATUS    RESTARTS   AGE
redis-76d7657885-lnmps   1/1       Running   0          8s

添加nginx

$ more rs.yaml
apiVersion: extensions/v1beta1
kind: ReplicaSet
metadata:
  name: nginx
spec:
  replicas: 2
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      name: nginx-pod
      labels:
        app: nginx
    spec:
      containers:
      - name: nginx-containers
        image: nginx

$ kubectl create -f rs.yaml 

replicaset "nginx" created

$ kubectl get replicaset

NAME               DESIRED   CURRENT   READY     AGE

nginx              2         1         1         7s

redis-76d7657885   1         1         1         50m

$ kubectl get pods

NAME                     READY     STATUS    RESTARTS   AGE

nginx-dr4t9              1/1       Running   0          33s

redis-76d7657885-lnmps   1/1       Running   0          6m

测试删除后自动回复：

$ kubectl delete pods nginx-dr4t9
pod "nginx-dr4t9" deleted
$ kubectl get pods
NAME                     READY     STATUS              RESTARTS   AGE
nginx-g8spl              1/1       Running             0          3s
nginx-mflm7              0/1       ContainerCreating   0          2s
redis-76d7657885-lnmps   1/1       Running             0          8m
$ kubectl get pods
NAME                     READY     STATUS    RESTARTS   AGE
nginx-g8spl              1/1       Running   0          6s
nginx-mflm7              1/1       Running   0          5s
redis-76d7657885-lnmps   1/1       Running   0          8m

选择label：

$ kubectl get pods -l app=nginx
NAME          READY     STATUS    RESTARTS   AGE
nginx-g8spl   1/1       Running   0          3h
nginx-mflm7   1/1       Running   0          3h

添加label：

$ kubectl get pods -Lapp
NAME                     READY     STATUS    RESTARTS   AGE       APP
nginx-g8spl              1/1       Running   0          3h        nginx
nginx-mflm7              1/1       Running   0          3h        nginx
redis-76d7657885-lnmps   1/1       Running   0          3h        
$ kubectl label pods redis-76d7657885-lnmps app=redis
pod "redis-76d7657885-lnmps" labeled
$ kubectl get pods -Lapp
NAME                     READY     STATUS    RESTARTS   AGE       APP
nginx-g8spl              1/1       Running   0          3h        nginx
nginx-mflm7              1/1       Running   0          3h        nginx
redis-76d7657885-lnmps   1/1       Running   0          3h        redis

添加多个label，显示所有label

$ kubectl label pods nginx-g8spl foo=bar
pod "nginx-g8spl" labeled
$ kubectl get pods --show-labels
NAME                     READY     STATUS    RESTARTS   AGE       LABELS
nginx-g8spl              1/1       Running   0          3h        app=nginx,foo=bar
nginx-mflm7              1/1       Running   0          3h        app=nginx
redis-76d7657885-lnmps   1/1       Running   0          3h        app=redis,k8s-app=redis,pod-template-hash=3283213441

思考：如果此时overwrite 一个app=nginx的pod会有什么后果：

$ kubectl label pods nginx-g8spl app=foobar --overwrite
pod "nginx-g8spl" labeled
$ kubectl get pods --show-labels
NAME                     READY     STATUS    RESTARTS   AGE       LABELS
nginx-g8spl              1/1       Running   0          3h        app=foobar,foo=baz
nginx-mflm7              1/1       Running   0          3h        app=nginx
nginx-mlhjs              1/1       Running   0          1m        app=nginx
redis-76d7657885-lnmps   1/1       Running   0          3h        app=redis,k8s-app=redis,pod-template-hash=3283213441

答：会额外多生成一个nginx pod，因为replicaset为label app=nginx定义的数量是2，所以它会始终维护数量为2

添加service：

$ cat svc.yaml 
apiVersion: v1
kind: Service
metadata:
  name: nginx
spec:
  ports:
  - name: main
    port: 80
  selector:
    app: nginx
  type: NodePort

$ kubectl create -f svc.yaml

service "nginx" created

查看并启动服务，会在浏览器中看到nginx的欢迎界面，port number是30978

$ kubectl get svc
NAME         TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)        AGE
kubernetes   ClusterIP   10.96.0.1        <none>        443/TCP        14d
nginx        NodePort    10.110.210.218   <none>        80:30978/TCP   5s

$ minikube service nginx

  $ kubectl get endpoints

NAME         ENDPOINTS                     AGE

kubernetes   10.0.2.15:8443                14d

nginx        172.17.0.6:80,172.17.0.7:80   1h

下面来个牛逼的操作，scale：

$ kubectl scale rs nginx --replicas=5
replicaset "nginx" scaled
$ kubectl get rs
NAME               DESIRED   CURRENT   READY     AGE
nginx              5         5         5         5h
redis-76d7657885   1         1         1         5h
$ kubectl get pods
NAME                     READY     STATUS    RESTARTS   AGE
nginx-d8n5h              1/1       Running   0          13s
nginx-g8spl              1/1       Running   0          5h
nginx-mflm7              1/1       Running   0          5h
nginx-mlhjs              1/1       Running   0          1h
nginx-nc5z7              1/1       Running   0          13s
nginx-t228t              1/1       Running   0          13s
redis-76d7657885-lnmps   1/1       Running   0          5h

我已经create了一个busybox，下面通过busybox来观察nginx：

$ kubectl exec -ti busybox-7c6c97f7cb-s6trg -- /bin/sh
/ # 
/ # nslookup nginx
Server:    10.96.0.10
Address 1: 10.96.0.10 kube-dns.kube-system.svc.cluster.local

Name:      nginx
Address 1: 10.110.210.218 nginx.default.svc.cluster.local

访问nginx：

/ # wget 10.110.210.218
Connecting to 10.110.210.218 (10.110.210.218:80)
index.html           100% |*************************************************************************************************************************|   612   0:00:00 ETA
/ # head index.html 
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
         35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }

好了，今天到此为止，更多内容下回分解