ingress-0.30.0

1. 部署ingress

1. 从github上下载ingress的部署文件

https://github.com/kubernetes/ingress-nginx

wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/nginx-0.30.0/deploy/static/mandatory.yaml

2. 修改 mandatory.yaml

https://gitee.com/chen1219_1/k8s-install/raw/master/ingress/mandatory.yaml

• 修改的部分

3. 下载 ingress-controller 镜像

docker pull siriuszg/nginx-ingress-controller:0.30.0
docker tag siriuszg/nginx-ingress-controller:0.30.0 harbor.od.com/k8s/nginx-ingress-controller:0.30.0
docker push harbor.od.com/k8s/nginx-ingress-controller:0.30.0

4. 编写一个nginx 的daemon nginxdaemon.yaml

https://gitee.com/chen1219_1/k8s-install/raw/master/ingress/nginxdaemon.yaml

5. 部署ingress-controller 和 nginx daemon

kubectl apply -f mandatory.yaml nginxdaemon.yaml 

6. 访问 nginx daemon的service

kubectl get svc

curl 10.47.224.97

7. 通过ingress 访问

• 查看ingress

kubectl get ing

• 查看ingress-controller 所在的机器

kubectl get all -n ingress-nginx -owide

• 本地配置dns (C:WindowsSystem32driversetchosts)

192.168.31.40 test.od.com

• 访问 test.od.com

8. ingress 配置https

8.1 下载镜像

docker pull wangyanglinux/myapp:v3
docker tag wangyanglinux/myapp:v3 harbor.od.com/public/wangyanglinux/myapp:v3
docker push harbor.od.com/public/wangyanglinux/myapp:v3

8.2 准备证书

mkdir https
cd https
openssl req -x509 -sha256 -nodes -days 365 -newkey rsa:2048 -keyout tls.key -out tls.crt -subj "/CN=nginxsvc/0=nginxsvc"
kubectl create secret tls tls-secret --key tls.key --cert tls.crt

8.3 准备资源

• nginxdaemon4https.yaml

  https://gitee.com/chen1219_1/k8s-install/raw/master/ingress/https/nginxdaemon4https.yaml

8.4 部署资源

kubectl apply -f nginxdaemon4https.yaml

8.5 访问 service

kubectl get svc

curl 10.47.116.140

8.6 通过ingress访问 test4https.od.com

kubectl get ing

• 设置本地dns解析

  因为dashboard是跑在ingress上， 域名所对应的ip设置成ingress-controller 所在的ip地址

kubectl get all -n ingress-nginx -owide

• 本地配置dns (C:WindowsSystem32driversetchosts)

192.168.31.40 test4https.od.com

• 浏览器输入 https://test4https.od.com

9. nginx 配置 BasicAuth

9.1 安装httpd

mkdir basicauth
cd basicauth/
yum -y install httpd

9.2 创建一个用户名为Jerry的密码文件auth

htpasswd -c auth Jerry

9.3 创建secret

kubectl create secret generic basic-auth --from-file=auth

9.4 准备资源

https://gitee.com/chen1219_1/k8s-install/raw/master/ingress/basicauth/nginxdaemon4basicauth.yaml

wget https://gitee.com/chen1219_1/k8s-install/raw/master/ingress/basicauth/nginxdaemon4basicauth.yaml

9.5 部署资源

kubectl apply -f nginxdaemon4basicauth.yaml

9.6 访问service

kubectl get svc

curl 10.47.208.187

9.7 ingress访问

• 本地配置dns (C:WindowsSystem32driversetchosts)

192.168.31.40 auth.od.com

• 访问 auth.od.com

输入 用户名Jerry 和 密码

10. 实现rewrite

10.1 创建资源

mkdir rewrite
cd rewrite/

• 创建资源 nginxdaemon.yaml

  https://gitee.com/chen1219_1/k8s-install/raw/master/ingress/rewrite/nginxdaemon.yaml

wget https://gitee.com/chen1219_1/k8s-install/raw/master/ingress/rewrite/nginxdaemon.yaml

10.2 部署资源

kubectl apply -f nginxdaemon.yaml

10.3 访问service

kubectl get svc

curl 10.47.134.140

10.4 ingress访问

• 本地配置dns (C:WindowsSystem32driversetchosts)

192.168.31.40 rewrite.od.com

• 访问 rewrite.od.com, 会跳转到 https://test4https.od.com/

2. 自定义ingress

2.1 部署ingress-controller

2.1.1 创建资源

https://gitee.com/chen1219_1/k8s-install/raw/master/ingress/custom-ingress/mandatory4custom.yaml

• 需要修改的部分

14	  name: nginx-configuration4public
24	  name: tcp-services4public
34	  name: udp-services4public

109	apiVersion: rbac.authorization.k8s.io/v1beta1
110	kind: Role
111	metadata:
112	  name: nginx-ingress-role
113	  namespace: ingress-nginx
114	  labels:
115	    app.kubernetes.io/name: ingress-nginx
116	    app.kubernetes.io/part-of: ingress-nginx
117	rules:
118	  - apiGroups:
119	      - ""
120	    resources:
121	      - configmaps
122	      - pods
123	      - secrets
124	      - namespaces
125	    verbs:
126	      - get
127	  - apiGroups:
128	      - ""
129	    resources:
130	      - configmaps
131	    resourceNames:
132	      # Defaults to "<election-id>-<ingress-class>"
133	      # Here: "<ingress-controller-leader>-<nginx>"
134	      # This has to be adapted if you change either parameter
135	      # when launching the nginx-ingress-controller.
136	      - "ingress-controller-leader-nginx"
137	      - "ingress-controller-leader-intranet4public"

191	apiVersion: apps/v1
192	kind: Deployment
193	metadata:
194	  name: ingress-controller4public
195	  namespace: ingress-nginx
196	  labels:
197	    app.kubernetes.io/name: ingress-nginx
198	    app.kubernetes.io/part-of: ingress-nginx
199	spec:
200	  replicas: 1
201	  selector:
202	    matchLabels:
203	      app.kubernetes.io/name: ingress-nginx
204	      app.kubernetes.io/part-of: ingress-nginx
205	  template:
206	    metadata:
207	      labels:
208	        app.kubernetes.io/name: ingress-nginx
209	        app.kubernetes.io/part-of: ingress-nginx
210	      annotations:
211	        prometheus.io/port: "10254"
212	        prometheus.io/scrape: "true"
213	    spec:
214	      hostNetwork: true
215	      # wait up to five minutes for the drain of connections
216	      terminationGracePeriodSeconds: 300
217	      serviceAccountName: nginx-ingress-serviceaccount
218	      nodeSelector:
219	        kubernetes.io/os: linux
220	      containers:
221	        - name: ingress-controller4public
222	          image: harbor.od.com/k8s/nginx-ingress-controller:0.30.0 
223	          args:
224	            - /nginx-ingress-controller
225	            - --configmap=$(POD_NAMESPACE)/nginx-configuration
226	            - --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
227	            - --udp-services-configmap=$(POD_NAMESPACE)/udp-services
228	            - --publish-service=$(POD_NAMESPACE)/ingress-nginx
229	            - --annotations-prefix=nginx.ingress.kubernetes.io
230	            - --ingress-class=intranet4public

2.1.2 部署资源

kubectl apply -f mandatory4custom.yaml

2.1.3 查看部署的ingress-controller的资源

kubectl get all -n ingress-nginx -owide

2.2 部署kubernetes-dashboard

2.2.3 下载kubernetes-dashboard 部署文件recommended.yaml 并将镜像的地址改为本地镜像的地址

wget https://raw.githubusercontent.com/kubernetes/dashboard/v2.0.3/aio/deploy/recommended.yaml
mv recommended.yaml dashboard.yaml

2.2.4 下载镜像

docker pull kubernetesui/dashboard:v2.0.3
docker tag kubernetesui/dashboard:v2.0.3 harbor.od.com/k8s/dashboard:v2.0.3
docker push harbor.od.com/k8s/dashboard:v2.0.3

docker pull kubernetesui/metrics-scraper:v1.0.4
docker tag kubernetesui/metrics-scraper:v1.0.4 harbor.od.com/k8s/metrics-scraper:v1.0.4
docker push harbor.od.com/k8s/metrics-scraper:v1.0.4

2.2.5 修改yaml文件

https://gitee.com/chen1219_1/k8s-install/raw/master/ingress/custom-ingress/dashboard.yaml

• 注释掉Dashboard Secret ，不然后面访问显示网页不安全，证书过期，我们自己生成证书

• 将镜像修改为镜像仓库地址

• 添加ingress配置

kind: Ingress
apiVersion: extensions/v1beta1
metadata:
  name: kubernetes-dashboard-ingress
  namespace: kubernetes-dashboard
  annotations:
    nginx.ingress.kubernetes.io/backend-protocol: HTTPS
    kubernetes.io/ingress.class: intranet4public 
spec:
  rules:
    - host: k8s-dashboard.paic.com.cn
      http:
        paths: 
          - path: /
            backend:
              serviceName: kubernetes-dashboard
              servicePort: 443

• 生成新的secret

  这里的secret必须在kubernetes-dashboard 名称空间生成, 否则dashboard会起不来, dashboard是启动在kubernetes-dashboard 这个名称空间, 所以secret 也必须在这个空间生成

mkdir key && cd key
openssl genrsa -out dashboard.key 2048
openssl req -new -out dashboard.csr -key dashboard.key -subj '/CN=192.168.31.10'
openssl x509 -req -in dashboard.csr -signkey dashboard.key -out dashboard.crt
kubectl create secret generic kubernetes-dashboard-certs --from-file=dashboard.key --from-file=dashboard.crt -n kubernetes-dashboard

• 部署dashboard

kubectl apply -f dashboard.yaml

2.2.6 设置权限文件

• admin-user.yaml

CopyapiVersion: v1
kind: ServiceAccount
metadata:
  name: admin-user
  namespace: kube-system

• admin-user-role-binding.yaml

CopyapiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: admin-user
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: cluster-admin
subjects:
- kind: ServiceAccount
  name: admin-user
  namespace: kube-system

• 部署权限文件

kubectl create -f admin-user.yaml 
kubectl create -f admin-user-role-binding.yaml

2.2.7 访问dashboard

• 设置本地dns解析

  因为dashboard是跑在ingress上， 域名所对应的ip设置成ingress-controller 所在的ip地址

  在ingress中指定了 kubernetes.io/ingress.class: intranet4public , 所以dashboard会跑在定义了- --ingress-class=intranet4public 的ingress-controller这台机器上

kubectl get all -n ingress-nginx -owide

• 本地配置dns (C:WindowsSystem32driversetchosts)

192.168.31.41 k8s-dashboard.paic.com.cn

• 访问 k8s-dashboard.paic.com.cn, 需要等待1-2分钟才能出来页面

• master 上查看token

kubectl describe secret `kubectl get secret -n kube-system |grep admin |awk '{print $1}'` -n kube-system |grep ^token|awk '{print $2}'