019.Kubernetes二进制集群存储longhorn

一 Longhorn存储部署

1.1 Longhorn概述

Longhorn是用于Kubernetes的开源分布式块存储系统。

提示：更多介绍参考：https://github.com/longhorn/longhorn。

1.2 Longhorn部署

[root@master01 ~]# cd /opt/k8s/work/
[root@master01 work]# source /root/environment.sh
[root@master01 work]# for all_ip in ${ALL_IPS[@]}
  do
    echo ">>> ${all_ip}"
    ssh root@${all_ip} "yum -y install iscsi-initiator-utils &"
  done

提示：本步骤操作仅需要在master01节点操作。

[root@master01 work]# mkdir longhorn
[root@master01 work]# cd longhorn/
[root@master01 longhorn]# wget 
https://raw.githubusercontent.com/longhorn/longhorn/master/deploy/longhorn.yaml
[root@master01 longhorn]# vi longhorn.yaml

#……
---
kind: Service
apiVersion: v1
metadata:
  labels:
    app: longhorn-ui
  name: longhorn-frontend
  namespace: longhorn-system
spec:
  type: NodePort			#修改为nodeport
  selector:
    app: longhorn-ui
  ports:
  - port: 80
    targetPort: 8000
    nodePort: 30002
---
……
kind: DaemonSet
……
        imagePullPolicy: IfNotPresent
……
#……

[root@master01 longhorn]# kubectl apply -f longhorn.yaml
[root@master01 longhorn]# kubectl -n longhorn-system get pods -o wide

提示：若部署异常可删除重建，若出现无法删除namespace，可通过如下操作进行删除：

wget https://github.com/longhorn/longhorn/blob/master/uninstall/uninstall.yaml
rm -rf /var/lib/longhorn/
kubectl apply -f uninstall.yaml
kubectl delete -f longhorn.yaml

1.5 动态sc创建

提示：默认longhorn部署完成已创建一个sc，也可通过如下手动编写yaml创建。

[root@master01 longhorn]# kubectl get sc
NAME                   PROVISIONER             RECLAIMPOLICY   VOLUMEBINDINGMODE      ALLOWVOLUMEEXPANSION   AGE
……
longhorn               driver.longhorn.io      Delete          Immediate              true                   15m
[root@master01 longhorn]# vi longhornsc.yaml

kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: longhornsc
provisioner: rancher.io/longhorn
parameters:
  numberOfReplicas: "3"
  staleReplicaTimeout: "30"
  fromBackup: ""

[root@master01 longhorn]# kubectl create -f longhornsc.yaml

1.6 测试PV及PVC

[root@master01 longhorn]# vi longhornpod.yaml

apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: longhorn-pvc
spec:
  accessModes:
    - ReadWriteOnce
  storageClassName: longhorn
  resources:
    requests:
      storage: 2Gi
---
apiVersion: v1
kind: Pod
metadata:
  name: longhorn-pod
  namespace: default
spec:
  containers:
  - name: volume-test
    image: nginx:stable-alpine
    imagePullPolicy: IfNotPresent
    volumeMounts:
    - name: volv
      mountPath: /data
    ports:
    - containerPort: 80
  volumes:
  - name: volv
    persistentVolumeClaim:
      claimName: longhorn-pvc

[root@master01 longhorn]# kubectl apply -f longhornpod.yaml
[root@master01 longhorn]# kubectl get pods -o wide
[root@master01 longhorn]# kubectl get pvc
[root@master01 longhorn]# kubectl get pv

提示：本步骤操作仅需要在master01节点操作。

1.7 Ingress暴露Longhorn

[root@master01 longhorn]# yum -y install httpd-tools
[root@master01 longhorn]# htpasswd -c auth xhy			#创建用户名和密码

提示：也可通过如下命令创建：

[root@master01 longhorn]# kubectl -n longhorn-system create secret generic longhorn-basic-auth --from-file=auth

[root@master01 longhorn]# vi longhorn-ingress.yaml		#创建ingress规则

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  name: longhorn-ingress
  namespace: longhorn-system
  annotations:
    kubernetes.io/ingress.class: "nginx"
    nginx.ingress.kubernetes.io/auth-type: basic
    nginx.ingress.kubernetes.io/auth-secret: longhorn-basic-auth
    nginx.ingress.kubernetes.io/auth-realm: 'Authentication Required '
spec:
  rules:
  - host: longhorn.odocker.com
    http:
      paths:
      - path: /
        backend:
          serviceName: longhorn-frontend
          servicePort: 80

[root@master01 longhorn]# kubectl apply -f longhorn-ingress.yaml

提示：本步骤操作仅需要在master01节点操作。

1.8 确认验证

浏览器访问：longhorn.odocker.com，并输入账号和密码。

登录查看。

附加 Helm部署

附0.1 helm安装

Helm 是 Kubernetes 的软件包管理工具，此处作为建议项部署，更多helm参考《053.集群管理-Helm工具》。

[root@master01 ~]# cd /opt/k8s/work/
[root@master01 work]# mkdir /opt/k8s/work/helm
[root@master01 work]# source /root/environment.sh
[root@master01 work]# wget http://down.linuxsb.com:8888/helm-v3.2.3-linux-amd64.tar.gz
[root@master01 work]# tar -zxvf helm-v3.2.3-linux-amd64.tar.gz -C /opt/k8s/work/helm
[root@master01 work]# for master_ip in ${MASTER_IPS[@]}
  do
    echo ">>> ${master_ip}"
    scp -rp /opt/k8s/work/helm/linux-amd64/helm root@${master_ip}:/opt/k8s/bin/
    ssh root@${master_ip} "chmod +x /opt/k8s/bin/*"
    ssh root@${master_ip} "helm version"
    ssh root@${master_ip} "echo 'source <(helm completion bash)' >> $HOME/.bashrc"
  done

附0.2 helm安装

[root@master01 work]# helm repo add brigade https://brigadecore.github.io/charts
[root@master01 work]# helm repo add stable https://kubernetes-charts.storage.googleapis.com/				#添加官方repo
[root@master01 work]# helm repo add bitnami https://charts.bitnami.com/bitnami
[root@master01 work]# helm repo list			#查看repo

提示：本步骤操作仅需要在master01节点操作。