使用helm进行kubernetes包管理

1. 安装helm package

https://github.com/helm/helm/blob/master/LICENSE

2. 将 helm 配置到环境变量

3. 使用helm的前提是安装了kubectl

4. 使用命令查看kubectl集群配置

如下所示表示与集群的通信正常（如果提示集群不可访问，可根据博文设置：https://www.cnblogs.com/miaoying/p/11193621.html）

$ kubectl cluster-info
Kubernetes master is running at https://192.168.1.164:6443
KubeDNS is running at https://192.168.1.164:6443/api/v1/namespaces/kube-system/services/kube-dns:dns/proxy
kubernetes-dashboard is running at https://192.168.1.164:6443/api/v1/namespaces/kube-system/services/https:kubernetes-dashboard:https/proxy

To further debug and diagnose cluster problems, use 'kubectl cluster-info dump'.

5. 查看k8s的client, server两端版本是否一致

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"10", GitVersion:"v1.10.3", GitCommit:"2bba0127d85d5a46ab4b778548be28623b32d0b0", GitTreeState:"clean", BuildDate:"2018-05-21T09:17:39Z", GoVersion:"go1.9.3", Compiler:"gc", Platform:"windows/amd64"}
Server Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.0", GitCommit:"e8462b5b5dc2584fdcd18e6bcfe9f1e4d970a529", GitTreeState:"clean", BuildDate:"2019-06-19T16:32:14Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"}

一开始我的client端版本是1.10.3的，server端的是1.15.0的，导致本地使用helm install包时一直抛异常，比如：

Error: release config failed: Deployment in version "v1beta1" cannot be handled as a Deployment: v1beta1.Deployment.Status: v1beta1.DeploymentStatus.Conditions: []v1beta1.DeploymentCondition: v1beta1.DeploymentCondition.Status: ReadString: expects " or n, but found t, error found in #10 byte of ...|"status":true,"type"|..., bigger context ...|ty","reason":"MinimumReplicasAvailable","status":true,"type":"Available"}],"observedGeneration":5,"r|...

原因是client端的版本与server端的不一致，导致两个端对配置文件的理解不一样，且有些配置属性低版本的不支持。

6. 将client、server端的版本升级成一致的：1.15.0

下载该版本的client端

https://dl.k8s.io/v1.15.0/kubernetes-client-windows-amd64.tar.gz

查看kubectl位置

$ which kubectl
/c/Program Files/Docker/Docker/Resources/bin/kubectl

发现kubectl在docker里（由于我的kubectl工具是安装docker for windows 时自带的，内置在docker里面）

为了使每次使用kubectl时，能够用1.15.0版本的，需要在环境变量里面对kubectl重新进行配置，我是直接配置在系统环境变量的Path里面，且将它的位置上移到最前面

另开一个终端，再次查看kubectl的位置，如下所示：

$ which kubectl
/f/kubernetes/install-1.15.0/kubectl

查看kubectl的版本信息（此时的版本信息是统一的了）

$ kubectl version
Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.0", GitCommit:"e8462b5b5dc2584fdcd18e6bcfe9f1e4d970a529", GitTreeState:"clean", BuildDate:"2019-06-19T16:40:16Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"windows/amd64"}
Server Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.0", GitCommit:"e8462b5b5dc2584fdcd18e6bcfe9f1e4d970a529", GitTreeState:"clean", BuildDate:"2019-06-19T16:32:14Z", GoVersion:"go1.12.5", Compiler:"gc", Platform:"linux/amd64"}

7. 查看当前kubernetes上下文

$ kubectl config current-context
kubernetes-admin@kubernetes

8. 初始化helm且安装tiller

helm init

由于 Helm 默认会去 storage.googleapis.com 拉取镜像，如果你当前执行的机器不能访问该域名的话可以使用以下命令来安装

helm init --client-only --stable-repo-url https://aliacs-app-catalog.oss-cn-hangzhou.aliyuncs.com/charts/
helm repo add incubator https://aliacs-app-catalog.oss-cn-hangzhou.aliyuncs.com/charts-incubator/
helm repo update

创建服务端

helm init --service-account tiller --upgrade -i registry.cn-hangzhou.aliyuncs.com/google_containers/tiller:v2.14.2  --stable-repo-url https://kubernetes.oss-cn-hangzhou.aliyuncs.com/charts

查看tiller

$ kubectl -n kube-system get pods|grep tiller
tiller-deploy-6bb85c7cb7-jhr65          1/1     Running   1          34h

9. 构建一个chart

我构建的chart的目录结构如下：

|-- dateway
|---- templates               # 【可选】 部署文件模板目录，模板使用的值来自values.yaml和tiller提供的值
|-----|-- deployment.yaml     
|---- Charts.yaml             # yaml文件，用于描述chart的基本信息，包括名字版本等
|---- values.yaml             # chart的默认配置文件

配置服务charts文件（注意yaml文件格式），可通过拉取原先已有的部署项目的配置文件：比如我是拉取curl的配置文件作为样例 kubectl get Deployment curl -o yaml > curl.yaml

（yaml格式是否合法，可通过 http://www.bejson.com/validators/yaml_editor/ 进行检验）

deployment.yaml

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  annotations:
    deployment.kubernetes.io/revision: "1"
  creationTimestamp: 2019-07-15T09:36:17Z
  generation: 1
  labels:
    app: gateway
  name: gateway
  namespace: rubikt
  resourceVersion: "1544"
  selfLink: /apis/extensions/v1beta1/namespaces/rubikt/deployments/gateway
  uid: a0ca862b-4a12-11e9-80d3-0050569b47c5
spec: 
  progressDeadlineSeconds: 600
  replicas: 1
  revisionHistoryLimit: 10
  selector: 
    matchLabels: 
      app: gateway
  strategy: 
    rollingUpdate: 
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template: 
    metadata: 
      creationTimestamp: null
      labels: 
        app: gateway
    spec: 
      containers: 
      - image: 192.168.2.50/health/gateway:1.5.1-SNAPSHOT
        ports:  
        - {containerPort: 8080, protocol: TCP}
        env:   
        - {name: JAVA_OPTS, value: '-Denv=dev -Ddev_meta=http://apollo.dev.zwjk.com -Dapp.id=health-gateway -server -Xmx768m'}
        - {name: CONSUL_URL, value: 10.109.36.166}
        - {name: CONSUL_HOST, value: 10.100.39.12}
        - {name: CONSUL_PORT, value: '8500'}
        terminationMessagePath: /dev/termination-log
        name: gateway
        resources:
          limits:
            cpu: 200m
            memory: 768Mi
          requests:
            cpu: 50m
            memory: 512Mi
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      securityContext: {}
      terminationGracePeriodSeconds: 30
      imagePullPolicy: Always
      imagePullSecrets:
      - {name: ccr.ccs.tencentyun.com.key}
      
status: 
  availableReplicas: 1
  conditions: 
  - lastTransitionTime: 2019-07-15T09:36:17Z
    lastUpdateTime: 2019-07-15T09:43:17Z
    message: ReplicaSet "gateway-6bf6db5c4f" has successfully progressed.
    reason: NewReplicaSetAvailable
    status: "True"
    type: Progressing
  - lastTransitionTime: 2019-07-15T09:43:59Z
    lastUpdateTime: 2019-07-15T09:43:59Z
    message: Deployment has minimum availability.
    reason: MinimumReplicasAvailable
    status: "True"
    type: Available
  observedGeneration: 1
  readyReplicas: 1
  replicas: 1
  updatedReplicas: 1

service.yaml

kind: Service
apiVersion: v1
metadata:
  name: gateway
  namespace: rubikt
  selfLink: /api/v1/namespaces/rubikt/services/gateway
  uid: 43fd6588-4a14-11e9-80d3-0050569b47c5
  resourceVersion: '103503'
  labels: {app: gateway}
spec:
  ports:
  - {protocol: TCP, port: 8080, targetPort: 8080, nodePort: 30013}
  selector: {app: gateway}
  type: NodePort
  sessionAffinity: None
status:
  loadBalancer: {}

Chart.yaml

name: gateway
version: "v1.0"
description: gateway

values.yaml

docker:
  registry: 192.168.2.50/health/gateway:1.5.1-SNAPSHOT
  pullPolicy: Always
core:
  replicas: 1 
config:
  version: "v1.0"

10. 使用命令来验证chart配置有效性

该输出中含模板的变量配置和最终渲染的yaml文件

（当使用kubernetes部署应用时，实际上是将templates渲染成最终的kubernetes能够识别的yaml格式）

$ helm install --dry-run --debug gateway
[debug] Created tunnel using local port: '56560'

[debug] SERVER: "127.0.0.1:56560"

[debug] Original chart version: ""
[debug] CHART PATH: D:zhuojian-projects
ubik-Tk8sgateway

NAME:   snug-hare
REVISION: 1
RELEASED: Thu Jul 18 18:03:57 2019
CHART: gateway-v1.0
USER-SUPPLIED VALUES:
{}

COMPUTED VALUES:
config:
  version: v1.0
core:
  replicas: 1
docker:
  pullPolicy: Always
  registry: 192.168.2.50/health/gateway:1.5.1-SNAPSHOT

HOOKS:
MANIFEST:

---
# Source: gateway/templates/service.yaml
kind: Service
apiVersion: v1
metadata:
  name: gateway
  namespace: rubikt
  selfLink: /api/v1/namespaces/rubikt/services/gateway
  uid: 43fd6588-4a14-11e9-80d3-0050569b47c5
  resourceVersion: '103503'
  labels: {app: gateway}
spec:
  ports:
  - {protocol: TCP, port: 8080, targetPort: 8080, nodePort: 30013}
  selector: {app: gateway}
  type: NodePort
  sessionAffinity: None
status:
  loadBalancer: {}
---
# Source: gateway/templates/deployment.yaml
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  annotations:
    deployment.kubernetes.io/revision: "1"
  creationTimestamp: 2019-07-15T09:36:17Z
  generation: 1
  labels:
    app: gateway
  name: gateway
  namespace: rubikt
  resourceVersion: "1544"
  selfLink: /apis/extensions/v1beta1/namespaces/rubikt/deployments/gateway
  uid: a0ca862b-4a12-11e9-80d3-0050569b47c5
spec:
  progressDeadlineSeconds: 600
  replicas: 1
  revisionHistoryLimit: 10
  selector:
    matchLabels:
      app: gateway
  strategy:
    rollingUpdate:
      maxSurge: 25%
      maxUnavailable: 25%
    type: RollingUpdate
  template:
    metadata:
      creationTimestamp: null
      labels:
        app: gateway
    spec:
      containers:
      - image: 192.168.2.50/health/gateway:1.5.1-SNAPSHOT
        ports:
        - {containerPort: 8080, protocol: TCP}
        env:
        - {name: JAVA_OPTS, value: '-Denv=dev -Ddev_meta=http://apollo.dev.zwjk.com -Dapp.id=health-gateway -server -Xmx768m'}
        - {name: CONSUL_URL, value: 10.109.36.166}
        - {name: CONSUL_HOST, value: 10.100.39.12}
        - {name: CONSUL_PORT, value: '8500'}
        terminationMessagePath: /dev/termination-log
        name: gateway
        resources:
          limits:
            cpu: 200m
            memory: 768Mi
          requests:
            cpu: 50m
            memory: 512Mi
      dnsPolicy: ClusterFirst
      restartPolicy: Always
      securityContext: {}
      terminationGracePeriodSeconds: 30
      imagePullPolicy: Always
      imagePullSecrets:
      - {name: ccr.ccs.tencentyun.com.key}

status:
  availableReplicas: 1
  conditions:
  - lastTransitionTime: 2019-07-15T09:36:17Z
    lastUpdateTime: 2019-07-15T09:43:17Z
    message: ReplicaSet "gateway-6bf6db5c4f" has successfully progressed.
    reason: NewReplicaSetAvailable
    status: "True"
    type: Progressing
  - lastTransitionTime: 2019-07-15T09:43:59Z
    lastUpdateTime: 2019-07-15T09:43:59Z
    message: Deployment has minimum availability.
    reason: MinimumReplicasAvailable
    status: "True"
    type: Available
  observedGeneration: 1
  readyReplicas: 1
  replicas: 1
  updatedReplicas: 1

11. 安装Chart到kubernetes

helm install -n gateway gateway

现在gateway已经部署到集群上了，本地可执行命令查看实例

$ kubectl get pods --namespace rubikt
NAME                             READY   STATUS             RESTARTS   AGE
gateway-86584799f-kvvsh          1/1     Running            0          31h

12. 查看部署的release

$ helm list
NAME                    REVISION        UPDATED                         STATUS          CHART                           APP VERSION     NAMESPACE
gateway                 5               Thu Jul 18 11:14:58 2019        DEPLOYED        gateway-v1.0                                    default

当前release包含的资源有：Service、Deployment、Secret、PersistentVolumeClaim

查看Service：

$ kubectl get service --namespace rubikt
NAME      TYPE       CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
gateway   NodePort   10.109.149.47   <none>        8080:30013/TCP   25h

查看Deployment：

$ kubectl get Deployment --namespace rubikt
NAME            READY   UP-TO-DATE   AVAILABLE   AGE
gateway         1/1     1            1           47h

查看Secret：

$ kubectl get Secret --namespace rubikt
NAME                  TYPE                                  DATA   AGE
default-token-687fw   kubernetes.io/service-account-token   3      2d5h

查看PersistentVolumeClaim：

$ kubectl get PersistentVolumeClaim --namespace rubikt
No resources found.

 13. 配置升级

更新了配置文件之后，可以使用upgrade命令进行更新（以此做到应用升级）

$ helm upgrade config config --description config升级 --version 2.0
Release "config" has been upgraded.
LAST DEPLOYED: Tue Jul 16 17:05:02 2019
NAMESPACE: default
STATUS: DEPLOYED

RESOURCES:
==> v1/Pod(related)
NAME                     READY  STATUS             RESTARTS  AGE
config-6c7b9f6885-jkf6j  1/1    Running            0         3m55s
config-fcf75984-vrd2f    0/1    ContainerCreating  0         0s

==> v1beta1/Deployment
NAME    READY  UP-TO-DATE  AVAILABLE  AGE
config  1/1    1           1          23m

可以看到部署的config已经升级了一个版本，revision = 2

$ helm list
NAME                    REVISION        UPDATED                         STATUS          CHART                           APP VERSION     NAMESPACE
config                  2               Tue Jul 16 16:54:51 2019        DEPLOYED        config-v1.0                                     default

14. 版本回退

回退应用版本，先使用helm history命令查看变更记录

$ helm history config
REVISION        UPDATED                         STATUS          CHART           DESCRIPTION
1               Tue Jul 16 16:41:10 2019        SUPERSEDED      config-v1.0     Install complete
2               Tue Jul 16 16:54:51 2019        SUPERSEDED      config-v1.0     Upgrade complete
3               Tue Jul 16 17:01:08 2019        SUPERSEDED      config-v1.0     Rollback to 1
4               Tue Jul 16 17:05:02 2019        DEPLOYED        config-v1.0     config升级

回滚到reversion为2的版本

$ helm rollback config 2
Rollback was a success.

另外： 部署到k8s上之后，可能会出现的问题：（表明需要配置内存空间大小）