k8s平台集成kong ingress 布署konga集成ui
各ingress的对比，其他博客有详细对比
简单谈点体验
-
istio很强大，但过于复杂，个人用过一段时期，勉强算是熟悉了基本使用，但复杂是相对团队的，每个人都掌握不可能，团队大部分只是想简单的上线一个api,然后能快速cicd部署上线到k8s 公开访问罢了，istio的流程做的全透明化需要额外的工作，个人来不及搞，团队后续为了图方便，都不喜欢用，算是半废弃状态，目前服务治理的方案是consul
-
ambassador 也是很强大的方案，未深入使用，只用过一些rewrite规则，方案比较强大
-
nginx/traefik 这两个用过，只是很简单的使用，未使用过复杂功能
-
kong 本身相比其他ingress并没有太大的优点，支持一些常用的插件,ssl托管，账号认证token,ip白名单等，不过这些也不只是kong有，真正决定使用kong，主要是因为konga的存在
官方向导见
https://docs.konghq.com/kubernetes-ingress-controller/1.1.x/deployment/minikube/
个人的环境并不是minikube 而是线上的生产集群，只不过喜欢，先以minikube类的精简的方案为基准，再逐组件替换为线上的ha方案
相比deploy，个人目前倾向deploy,改为sts
-
改动 变更postgres类型
生产环境，建议使用postgres HA方案https://github.com/sorintlab/stolon 官方deploy为postgres,个人会替换为StatefulSet
apiVersion: apps/v1 kind: StatefulSet metadata: name: postgres namespace: kong spec: replicas: 1 selector: matchLabels: app: postgres serviceName: postgres template: metadata: labels: app: postgres spec: containers: - env: - name: POSTGRES_USER value: kong - name: POSTGRES_PASSWORD value: kong - name: POSTGRES_DB value: kong - name: PGDATA value: /var/lib/postgresql/data/pgdata image: postgres:9.5 name: postgres ports: - containerPort: 5432 volumeMounts: - mountPath: /var/lib/postgresql/data name: datadir subPath: pgdata terminationGracePeriodSeconds: 60 volumeClaimTemplates: - metadata: name: datadir spec: accessModes: - ReadWriteOnce resources: requests: storage: 1Gi -
官方只有kong ingress是不够的，需要布署konga,之所以用kong就是因为konga的存在
apiVersion: apps/v1 kind: Deployment metadata: labels: app: konga name: konga namespace: kong spec: replicas: 1 selector: matchLabels: app: konga template: metadata: labels: app: konga spec: containers: - env: - name: HOST value: 0.0.0.0 - name: PORT value: '80' - name: NODE_ENV value: production - name: DB_ADAPTER value: postgres - name: DB_HOST value: kong - name: DB_PORT value: '5432' - name: DB_USER value: kong - name: DB_PASSWORD value: kong - name: DB_DATABASE value: kong - name: DB_PG_SCHEMA value: kong - name: NO_AUTH value: 'true' image: pantsel/konga:0.14.9 name: konga ports: - containerPort: 80 --- apiVersion: v1 kind: Service metadata: name: konga namespace: kong spec: externalTrafficPolicy: Cluster ports: - name: konga port: 80 protocol: TCP targetPort: 80 selector: app: konga
初始化konga，默认konga不可执行，进入konga执行初始化db
./bin/konga.js -c prepare -a postgres -u postgresql://kong:kong@postgres:5432/konga
$ kb -n kong get pod
NAME READY STATUS RESTARTS AGE
ingress-kong-6b9544969-2pxwl 2/2 Running 0 125m
kong-migrations-6rshd 0/1 Completed 0 170m
postgres-767c99c648-fgd97 1/1 Running 0 20m
$ kb -n kong get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kong-proxy LoadBalancer 10.99.12.233 <pending> 80:30193/TCP,443:31473/TCP 170m
kong-validation-webhook ClusterIP 10.102.217.8 <none> 443/TCP 170m
postgres ClusterIP 10.105.201.5 <none> 5432/TCP 170m
暂时把 kong-proxy LoadBalancer 改为 NodePort
$ kb -n kong get svc
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
kong-proxy NodePort 10.99.12.233 <none> 80:30193/TCP,443:31473/TCP 171m
kong-validation-webhook ClusterIP 10.102.217.8 <none> 443/TCP 171m
postgres ClusterIP 10.105.201.5 <none> 5432/TCP 171m
验证访问
curl -i $PROXY_IP
HTTP/1.1 404 Not Found
Date: Tue, 30 Jun 2020 09:34:23 GMT
Content-Type: application/json; charset=utf-8
Connection: keep-alive
Content-Length: 48
X-Kong-Response-Latency: 1
Server: kong/2.0.4
{"message":"no Route matched with those values"}
验证真实地址,确认kong 集成成功
$ curl -i $PROXY_IP/foo
HTTP/1.1 200 OK
Content-Type: text/plain; charset=UTF-8
Transfer-Encoding: chunked
Connection: keep-alive
Date: Tue, 30 Jun 2020 09:34:40 GMT
Server: echoserver
X-Kong-Upstream-Latency: 2
X-Kong-Proxy-Latency: 4
Via: kong/2.0.4
Hostname: echo-599d77c5c7-jv8jl
Pod Information:
pod name: echo-599d77c5c7-jv8jl
pod namespace: default
pod IP: 192.168.63.51
Server values:
server_version=nginx: 1.12.2 - lua: 10010
Request Information:
client_address=192.168.111.254
method=GET
real path=/foo
query=
request_version=1.1
request_scheme=http
Request Headers:
accept=*/*
connection=keep-alive
user-agent=curl/7.29.0
x-forwarded-for=192.168.75.0
x-forwarded-port=8000
x-forwarded-proto=http
x-real-ip=192.168.75.0
Request Body:
-no body in request-
通过konga注册绑定k8s的kong
试了 nodeport ingress host,api 都失败 查看konga日志得
KongProxyController request error undefined
Sending 500 ("Server Error") response:
{
error: Error: self signed certificate
at TLSSocket.onConnectSecure (_tls_wrap.js:1474:34)
at TLSSocket.emit (events.js:310:20)
at TLSSocket.EventEmitter.emit (domain.js:482:12)
at TLSSocket._finishInit (_tls_wrap.js:917:8)
at TLSWrap.ssl.onhandshakedone (_tls_wrap.js:687:12) {
code: 'DEPTH_ZERO_SELF_SIGNED_CERT'
}
}
error: unexpected EOF
是konga 访问kong-admin-api的证书认证失败
查看到官方kong ingress 默认的配置是
spec:
containers:
- env:
- name: KONG_PROXY_LISTEN
value: 0.0.0.0:8000, 0.0.0.0:8443 ssl http2
- name: KONG_ADMIN_LISTEN
value: 127.0.0.1:8444 ssl
- name: KONG_STATUS_LISTEN
value: 0.0.0.0:8100
- name: KONG_DATABASE
value: postgres
- name: KONG_PG_HOST
value: stolon-proxy-service.default
- name: KONG_PG_PASSWORD
value: bia_miaozhen
- name: KONG_NGINX_WORKER_PROCESSES
value: "1"
- name: KONG_ADMIN_ACCESS_LOG
value: /dev/stdout
- name: KONG_ADMIN_ERROR_LOG
value: /dev/stderr
- name: KONG_PROXY_ERROR_LOG
value: /dev/stderr
image: kong:2.0
imagePullPolicy: IfNotPresent
KONG_ADMIN_LISTEN: 127.0.0.1:8444 ssl
更改 KONG_ADMIN_LISTEN 为 0.0.0.0:8444 ssl ，可以外部访问，但证书认证不通过
两个思路
- 1加证书认证
暂内网服务，为图简单，先不采用证书的方式
- 2通过http访问
先公开http服务，看kong的官方示例和文档
$ docker run -d --name kong
--link kong-database:kong-database
-e "KONG_DATABASE=postgres"
-e "KONG_PG_HOST=kong-database"
-e "KONG_CASSANDRA_CONTACT_POINTS=kong-database"
-e "KONG_PROXY_ACCESS_LOG=/dev/stdout"
-e "KONG_ADMIN_ACCESS_LOG=/dev/stdout"
-e "KONG_PROXY_ERROR_LOG=/dev/stderr"
-e "KONG_ADMIN_ERROR_LOG=/dev/stderr"
-e "KONG_ADMIN_LISTEN=0.0.0.0:8001, 0.0.0.0:8444 ssl"
-p 8000:8000
-p 8443:8443
-p 8001:8001
-p 8444:8444
kong
KONG_ADMIN_LISTEN是允许公开多个地址的，带ssl后缀的为https，不带ssl的 为http
我们额外公开8001做为KONG_ADMIN_LISTEN http服务，重启生效
- name: KONG_ADMIN_LISTEN
value: 0.0.0.0:8001, 0.0.0.0:8444 ssl
外部通过ingress 可以访问kong-admin-api.bia.com
http://ingress-kong-admin:8001/
截图和配置略有不同，个人的db是 stolon
kong ingress+konga 配置完毕
k8s kong ingress有一些集成的功能可以通过参数配置好，例如rewrite,下篇会谈
也可以k8s kong ingress 只注册一个标准的ingress，之后konga会同步显示，在页面上为该ingress配置各种插件
End