1、认识Ingress
1.1 什么是Ingress?
通常情况下,service和pod仅可在集群内部网络中通过IP地址访问。所有到达边界路由器的流量或被丢弃或被转发到其他地方。从概念上讲,可能像下面这样:
1
2
3
4
|
internet | ------------ [ Services ] |
Ingress是授权入站连接到达集群服务的规则集合。
1
2
3
4
5
|
internet | [ Ingress ] --|-----|-- [ Services ] |
你可以给Ingress配置提供外部可访问的URL、负载均衡、SSL、基于名称的虚拟主机等。用户通过POST Ingress资源到API server的方式来请求ingress。 Ingress controller负责实现Ingress,通常使用负载平衡器,它还可以配置边界路由和其他前端,这有助于以HA方式处理流量。
1.2 Ingress工作示意图
1.3先决条件
在使用Ingress resource之前,有必要先了解下面几件事情。Ingress是beta版本的resource,在kubernetes1.1之前还没有。你需要一个Ingress Controller来实现Ingress,单纯的创建一个Ingress没有任何意义。
GCE/GKE会在master节点上部署一个ingress controller。你可以在一个pod中部署任意个自定义的ingress controller。你必须正确地annotate每个ingress,比如 运行多个ingress controller 和 关闭glbc.
确定你已经阅读了Ingress controller的beta版本限制。在非GCE/GKE的环境中,你需要在pod中部署一个controller。
1.4 Ingress定义资源清单几个字段
- apiVersion: v1 版本
- kind: Ingress 类型
- metadata 元数据
- spec 期望状态
- backend: 默认后端,能够处理与任何规则不匹配的请求
- rules:用于配置Ingress的主机规则列表
- tls:目前Ingress仅支持单个TLS端口443
- status 当前状态
2、部署一个Ingress
(1)在gitlab上下载yaml文件,并创建部署
gitlab ingress-nginx项目:https://github.com/kubernetes/ingress-nginx
ingress安装指南:https://kubernetes.github.io/ingress-nginx/deploy/
因为需要拉取镜像,所以需要等一段时间
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
|
---下载需要的yaml文件 [root@master ingress-nginx] # wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/mandatory.yaml ---查询下载成功 [root@master ingress-nginx] # ls mandatory.yaml ---创建ingress [root@master ingress-nginx] # kubectl apply -f mandatory.yaml namespace /ingress-nginx created configmap /nginx-configuration created configmap /tcp-services created configmap /udp-services created serviceaccount /nginx-ingress-serviceaccount created clusterrole.rbac.authorization.k8s.io /nginx-ingress-clusterrole created role.rbac.authorization.k8s.io /nginx-ingress-role created rolebinding.rbac.authorization.k8s.io /nginx-ingress-role-nisa-binding created clusterrolebinding.rbac.authorization.k8s.io /nginx-ingress-clusterrole-nisa-binding created deployment.apps /nginx-ingress-controller created |
(2)如果是裸机,还需要安装service
1
2
3
|
[root@master ingress-nginx] # wget https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/provider/baremetal/service-nodeport.yaml [root@master ingress-nginx] # kubectl apply -f service-nodeport.yaml service /ingress-nginx created |
(3)验证
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
|
---查询生产的pod [root@master ~] # kubectl get pods -n ingress-nginx NAME READY STATUS RESTARTS AGE nginx-ingress-controller-648c7bb65b-df9qz 1 /1 Running 0 34m ---查询生产的svc [root@master ingress-nginx] # kubectl get svc -n ingress-nginx NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE ingress-nginx NodePort 10.109.244.123 <none> 80:30080 /TCP ,443:30443 /TCP 21s ---查询svc的详细信息 [root@master ~] # kubectl describe svc ingress-nginx -n ingress-nginx Name: ingress-nginx Namespace: ingress-nginx Labels: app.kubernetes.io /name =ingress-nginx app.kubernetes.io /part-of =ingress-nginx Annotations: kubectl.kubernetes.io /last-applied-configuration ={ "apiVersion" : "v1" , "kind" : "Service" , "metadata" :{ "annotations" :{}, "labels" :{ "app.kubernetes.io/name" : "ingress-nginx" , "app.kubernetes.io/part-of" :"ingres... Selector: app.kubernetes.io /name =ingress-nginx,app.kubernetes.io /part-of =ingress-nginx Type: NodePort IP: 10.111.143.90 Port: http 80 /TCP TargetPort: 80 /TCP NodePort: http 30080 /TCP Endpoints: 10.244.1.104:80 Port: https 443 /TCP TargetPort: 443 /TCP NodePort: https 30443 /TCP Endpoints: 10.244.1.104:443 Session Affinity: None External Traffic Policy: Cluster Events: <none> |
3、创建Ingress,代理到后端nginx服务
3.1 准备后端pod和service
(1)编写yaml文件,并创建
创建3个nginx服务的pod,并创建一个service绑定
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
|
[root@master ingress] # vim deploy-damo.yaml apiVersion: v1 kind: Service metadata: name: myapp namespace: default spec: selector: app: myapp release: canary ports: - name: http targetPort: 80 port: 80 --- apiVersion: apps /v1 kind: Deployment metadata: name: myapp-deploy namespace: default spec: replicas: 3 selector: matchLabels: app: myapp release: canary template: metadata: labels: app: myapp release: canary spec: containers: - name: myapp image: ikubernetes /myapp :v2 ports: - name: http containerPort: 80 [root@master ingress] # kubectl apply -f deploy-damo.yaml service /myapp created deployment.apps /myapp-deploy created |
(2)查询验证
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
|
[root@master ~] # kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.96.0.1 <none> 443 /TCP 146d myapp ClusterIP 10.103.137.126 <none> 80 /TCP 6s [root@master ~] # kubectl get pods NAME READY STATUS RESTARTS AGE myapp-deploy-67f6f6b4dc-2vzjn 1 /1 Running 0 14s myapp-deploy-67f6f6b4dc-c7f76 1 /1 Running 0 14s myapp-deploy-67f6f6b4dc-x79hc 1 /1 Running 0 14s [root@master ~] # kubectl describe svc myapp Name: myapp Namespace: default Labels: <none> Annotations: kubectl.kubernetes.io /last-applied-configuration ={ "apiVersion" : "v1" , "kind" : "Service" , "metadata" :{ "annotations" :{}, "name" : "myapp" , "namespace" : "default" }, "spec" :{ "ports" :[{ "name" : "http" , "port" :80,"targe... Selector: app=myapp,release=canary Type: ClusterIP IP: 10.103.137.126 Port: http 80 /TCP TargetPort: 80 /TCP Endpoints: 10.244.1.102:80,10.244.1.103:80,10.244.2.109:80 Session Affinity: None Events: <none> |
3.2 创建ingress,绑定后端nginx服务
(1)编写yaml文件,并创建
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
|
[root@master ingress] # vim ingress-myapp.yaml apiVersion: extensions /v1beta1 kind: Ingress metadata: name: ingress-myapp namespace: default spec: rules: - host: myapp.along.com http: paths: - path: backend: serviceName: myapp servicePort: 80 [root@master ingress] # kubectl apply -f ingress-myapp.yaml ingress.extensions /ingress-myapp created |
(2)查询验证
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
|
[root@master ~] # kubectl get ingress NAME HOSTS ADDRESS PORTS AGE ingress-myapp myapp.along.com 80 140d [root@master ~] # kubectl describe ingress ingress-myapp Name: ingress-myapp Namespace: default Address: Default backend: default-http-backend:80 (<none>) Rules: Host Path Backends ---- ---- -------- myapp.along.com myapp:80 (<none>) Annotations: kubectl.kubernetes.io /last-applied-configuration : { "apiVersion" : "extensions/v1beta1" , "kind" : "Ingress" , "metadata" :{ "annotations" :{}, "name" : "ingress-myapp" , "namespace" : "default" }, "spec" :{ "rules" :[{ "host" : "myapp.along.com" , "http" :{ "paths" :[{ "backend" :{ "serviceName" : "myapp" , "servicePort" :80}, "path" :null}]}}]}} Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal CREATE 37s nginx-ingress-controller Ingress default /ingress-myapp |
(3)在集群外,查询服务验证
① 可以先修改一下主机的hosts,因为不是公网域名
192.168.130.103 myapp.along.com
② 访问业务成功
4、创建Ingress,代理到后端tomcat服务
4.1 准备后端pod和service
(1)编写yaml文件,并创建
创建3个tomcat服务的pod,并创建一个service绑定
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
|
[root@master ingress] # vim tomcat-deploy.yaml apiVersion: v1 kind: Service metadata: name: tomcat namespace: default spec: selector: app: tomcat release: canary ports: - name: http targetPort: 8080 port: 8080 - name: ajp targetPort: 8009 port: 8009 --- apiVersion: apps /v1 kind: Deployment metadata: name: tomcat-deploy namespace: default spec: replicas: 3 selector: matchLabels: app: tomcat release: canary template: metadata: labels: app: tomcat release: canary spec: containers: - name: tomcat image: tomcat:8.5.37-jre8-alpine ports: - name: http containerPort: 8080 - name: ajp containerPort: 8009 [root@master ingress] # kubectl apply -f tomcat-deploy.yaml service /tomcat created deployment.apps /tomcat-deploy created |
(2)查询验证
1
2
3
4
5
6
7
8
9
|
[root@master ~] # kubectl get pods NAME READY STATUS RESTARTS AGE tomcat-deploy-97d6458c5-hrmrw 1 /1 Running 0 1m tomcat-deploy-97d6458c5-ngxxx 1 /1 Running 0 1m tomcat-deploy-97d6458c5-xchgn 1 /1 Running 0 1m [root@master ~] # kubectl get svc NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE kubernetes ClusterIP 10.96.0.1 <none> 443 /TCP 146d tomcat ClusterIP 10.98.193.252 <none> 8080 /TCP ,8009 /TCP 1m |
4.2 创建ingress,绑定后端tomcat服务
(1)编写yaml文件,并创建
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
|
[root@master ingress] # vim ingress-tomcat.yaml apiVersion: extensions /v1beta1 kind: Ingress metadata: name: ingress-tomcat namespace: default spec: rules: - host: tomcat.along.com http: paths: - path: backend: serviceName: tomcat servicePort: 8080 [root@master ingress] # kubectl apply -f ingress-tomcat.yaml ingress.extensions /ingress-tomcat created |
(2)查询验证
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
|
[root@master ~] # kubectl get ingress NAME HOSTS ADDRESS PORTS AGE ingress-myapp myapp.along.com 80 17m ingress-tomcat tomcat.along.com 80 6s [root@master ~] # kubectl describe ingress ingress-tomcat Name: ingress-tomcat Namespace: default Address: Default backend: default-http-backend:80 (<none>) Rules: Host Path Backends ---- ---- -------- tomcat.along.com tomcat:8080 (<none>) Annotations: kubectl.kubernetes.io /last-applied-configuration : { "apiVersion" : "extensions/v1beta1" , "kind" : "Ingress" , "metadata" :{ "annotations" :{}, "name" : "ingress-tomcat" , "namespace" : "default" }, "spec" :{ "rules" :[{ "host" : "tomcat.along.com" , "http" :{ "paths" :[{ "backend" :{ "serviceName" : "tomcat" , "servicePort" :8080}, "path" :null}]}}]}} Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal CREATE 17s nginx-ingress-controller Ingress default /ingress-tomcat |
(3)在集群外,查询服务验证
① 可以先修改一下主机的hosts,因为不是公网域名
192.168.130.103 tomcat.along.com
② 访问业务成功
4.3 使用https协议访问服务
4.3.1 创建证书、私钥和secret
(1)创建私钥
1
2
3
4
5
6
7
|
[root@master ingress] # openssl genrsa -out tls.key 2048 Generating RSA private key, 2048 bit long modulus .............................................+++ ...............+++ e is 65537 (0x10001) [root@master ingress] # ls *key tls.key |
(2)创建证书
1
2
3
|
[root@master ingress] # openssl req -new -x509 -key tls.key -out tls.crt -subj /C=CN/ST=Beijing/L=Beijing/O=DevOps/CN=tomcat.along.com [root@master ingress] # ls tls.* tls.crt tls.key |
(3)创建secret
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
|
[root@master ingress] # kubectl create secret tls tomcat-ingress-secret --cert=tls.crt --key=tls.key secret /tomcat-ingress-secret created [root@master ingress] # kubectl get secret NAME TYPE DATA AGE tomcat-ingress-secret kubernetes.io /tls 2 8s [root@master ingress] # kubectl describe secret tomcat-ingress-secret Name: tomcat-ingress-secret Namespace: default Labels: <none> Annotations: <none> Type: kubernetes.io /tls Data ==== tls.key: 1675 bytes tls.crt: 1294 bytes |
4.3.2 重新创建ingress,使用https协议绑定后端tomcat服务
(1)编写yaml文件,并创建
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
|
[root@master ingress] # vim ingress-tomcat-tls.yaml apiVersion: extensions /v1beta1 kind: Ingress metadata: name: ingress-tomcat-tls namespace: default spec: tls: - hosts: - tomcat.along.com secretName: tomcat-ingress-secret rules: - host: tomcat.along.com http: paths: - path: backend: serviceName: tomcat servicePort: 8080 |
(2)查询验证
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
|
[root@master ~] # kubectl get ingress NAME HOSTS ADDRESS PORTS AGE ingress-myapp myapp.along.com 80 34m ingress-tomcat tomcat.along.com 80 16m ingress-tomcat-tls tomcat.along.com 80, 443 8s [root@master ~] # kubectl describe ingress ingress-tomcat-tls Name: ingress-tomcat-tls Namespace: default Address: Default backend: default-http-backend:80 (<none>) TLS: tomcat-ingress-secret terminates tomcat.along.com Rules: Host Path Backends ---- ---- -------- tomcat.along.com tomcat:8080 (<none>) Annotations: kubectl.kubernetes.io /last-applied-configuration : { "apiVersion" : "extensions/v1beta1" , "kind" : "Ingress" , "metadata" :{ "annotations" :{}, "name" : "ingress-tomcat-tls" , "namespace" : "default" }, "spec" :{ "rules" :[{ "host" : "tomcat.along.com" , "http" :{ "paths" :[{ "backend" :{ "serviceName" : "tomcat" , "servicePort" :8080}, "path" :null}]}}], "tls" :[{ "hosts" :[ "tomcat.along.com" ], "secretName" : "tomcat-ingress-secret" }]}} Events: Type Reason Age From Message ---- ------ ---- ---- ------- Normal CREATE 14s nginx-ingress-controller Ingress default /ingress-tomcat-tls |
(3)在集群外,查询服务验证
使用https协议,访问业务成功