Kubernetes之Ingress自动化https

目录

• 1、前置条件
• 2、部署cert-manager
  • 2.1、创建一个namespace
  • 2.2、安装cert-manager
  • 2.3、测试
• 3、创建clusterissuer
• 4、为域名创建certificate
• 5、在ingress中引用对应的secret
• 6、自动化颁发证书

cert-manager 是一个云原生证书管理开源项目，用于在 Kubernetes 集群中提供 HTTPS 证书并自动续期，支持 Let’s Encrypt, HashiCorp Vault 这些免费证书的签发。在Kubernetes集群中，我们可以通过 Kubernetes Ingress 和 Let’s Encrypt 实现外部服务的自动化 HTTPS。

1、前置条件

在Kubernetes集群中使用 HTTPS 协议，需要一个证书管理器、一个证书自动签发服务，主要通过 Ingress 来发布 HTTPS 服务，因此需要Ingress Controller并进行配置，启用 HTTPS 及其路由。

本文环境：

• k8s v1.17.0
• Ingress Controller为nginx，且有对应暴露的公网ip地址

2、部署cert-manager

从cert-manager v0.11.0开始，Kubernetes的最低支持版本是v1.12.0。仍在运行Kubernetes v1.11或更低版本的用户应在安装cert-manager之前升级到受支持的版本。

cert-manager可以通过官方yaml安装或者通过helm快速安装，本文记录通过官方yaml安装的过程

2.1、创建一个namespace

# namespace.yaml 
---
apiVersion: v1
kind: Namespace
metadata:
  name: cert-manager

或者

kubectl create namespace cert-manager

2.2、安装cert-manager

官方的yaml地址为

https://github.com/jetstack/cert-manager/releases/download/v0.13.1/cert-manager.yaml

yaml中有三个镜像，分别为

• cert-manager-controller:v0.13.1
• cert-manager-cainjector:v0.13.1
• cert-manager-webhook:v0.13.1

默认是从quay.io获取镜像，如果quay.io的镜像无法获取，修改image为国内源，例如Azure中国的地址quay.azk8s.cn

部署，会在集群中创建一系列的crd资源

# kubectl apply -f cert-manager.yaml 
customresourcedefinition.apiextensions.k8s.io/certificaterequests.cert-manager.io created
customresourcedefinition.apiextensions.k8s.io/certificates.cert-manager.io created
customresourcedefinition.apiextensions.k8s.io/challenges.acme.cert-manager.io created
customresourcedefinition.apiextensions.k8s.io/clusterissuers.cert-manager.io created
customresourcedefinition.apiextensions.k8s.io/issuers.cert-manager.io created
customresourcedefinition.apiextensions.k8s.io/orders.acme.cert-manager.io created
namespace/cert-manager unchanged
serviceaccount/cert-manager-cainjector created
serviceaccount/cert-manager created
serviceaccount/cert-manager-webhook created
clusterrole.rbac.authorization.k8s.io/cert-manager-cainjector created
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-cainjector created
role.rbac.authorization.k8s.io/cert-manager-cainjector:leaderelection created
rolebinding.rbac.authorization.k8s.io/cert-manager-cainjector:leaderelection created
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-webhook:auth-delegator created
rolebinding.rbac.authorization.k8s.io/cert-manager-webhook:webhook-authentication-reader created
clusterrole.rbac.authorization.k8s.io/cert-manager-webhook:webhook-requester created
role.rbac.authorization.k8s.io/cert-manager:leaderelection created
rolebinding.rbac.authorization.k8s.io/cert-manager:leaderelection created
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-issuers created
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-clusterissuers created
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-certificates created
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-orders created
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-challenges created
clusterrole.rbac.authorization.k8s.io/cert-manager-controller-ingress-shim created
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-issuers created
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-clusterissuers created
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-certificates created
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-orders created
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-challenges created
clusterrolebinding.rbac.authorization.k8s.io/cert-manager-controller-ingress-shim created
clusterrole.rbac.authorization.k8s.io/cert-manager-view created
clusterrole.rbac.authorization.k8s.io/cert-manager-edit created
service/cert-manager created
service/cert-manager-webhook created
deployment.apps/cert-manager-cainjector created
deployment.apps/cert-manager created
deployment.apps/cert-manager-webhook created
mutatingwebhookconfiguration.admissionregistration.k8s.io/cert-manager-webhook created
validatingwebhookconfiguration.admissionregistration.k8s.io/cert-manager-webhook created

检查对应的pod状态

# kubectl get pods -n cert-manager 
NAME                                      READY   STATUS    RESTARTS   AGE
cert-manager-5cbcb9f4f5-7k6j4             1/1     Running   0          90s
cert-manager-cainjector-8df55567d-bspds   1/1     Running   0          90s
cert-manager-webhook-5d9c55bb4c-tmlck     1/1     Running   0          88s

2.3、测试

在正式使用前，先通过官方的示例做一个测试来确认正确设置了cert-manager并能够颁发基本证书类型
测试Webhook正常工作

# cat <<EOF > test-resources.yaml
apiVersion: v1
kind: Namespace
metadata:
  name: cert-manager-test
---
apiVersion: cert-manager.io/v1alpha2
kind: Issuer
metadata:
  name: test-selfsigned
  namespace: cert-manager-test
spec:
  selfSigned: {}
---
apiVersion: cert-manager.io/v1alpha2
kind: Certificate
metadata:
  name: selfsigned-cert
  namespace: cert-manager-test
spec:
  dnsNames:
    - example.com
  secretName: selfsigned-cert-tls
  issuerRef:
    name: test-selfsigned
EOF

创建测试资源

# kubectl apply -f test-resources.yaml
namespace/cert-manager-test created
issuer.cert-manager.io/test-selfsigned created
certificate.cert-manager.io/selfsigned-cert created

查新创建证书的状态。可能需要等待几秒钟，然后cert-manager才能处理证书请求

# kubectl -n cert-manager-test describe certificate selfsigned-cert
...
Spec:
  Dns Names:
    example.com
  Issuer Ref:
    Name:       test-selfsigned
  Secret Name:  selfsigned-cert-tls
Status:
  Conditions:
    Last Transition Time:  2020-03-05T10:01:06Z
    Message:               Certificate is up to date and has not expired
    Reason:                Ready
    Status:                True
    Type:                  Ready
  Not After:               2020-06-03T10:01:06Z
Events:
  Type    Reason        Age   From          Message
  ----    ------        ----  ----          -------
  Normal  GeneratedKey  61s   cert-manager  Generated a new private key
  Normal  Requested     61s   cert-manager  Created new CertificateRequest resource "selfsigned-cert-504566127"
  Normal  Issued        61s   cert-manager  Certificate issued successfully

清理测试资源

# kubectl delete -f test-resources.yaml 
namespace "cert-manager-test" deleted
issuer.cert-manager.io "test-selfsigned" deleted
certificate.cert-manager.io "selfsigned-cert" deleted

3、创建clusterissuer

为了配置cert-manager以开始颁发证书，必须先创建Issuer或ClusterIssuer资源。这些资源代表特定的签名机构，并详细说明如何满足证书请求。Issuer只能用来签发自己所在namespace下的证书，ClusterIssuer可以签发任意namespace下的证书，这里以ClusterIssuer为例创建一个签发机构

# cat clusterissuer.yaml
apiVersion: cert-manager.io/v1alpha2
kind: ClusterIssuer
metadata:
  name: letsencrypt-prod
spec:
  acme:
    server: https://acme-v02.api.letsencrypt.org/directory
    email: ssgeek@ssgeek.com
    privateKeySecretRef:
      name: letsencrypt-prod
    solvers:
    - http01:
        ingress:
          class: nginx

说明：

• metadata.name 创建的签发机构的名称，创建证书的时候会引用
• spec.acme.email 邮箱，证书快过期的时候会有邮件提醒，不过cert-manager会利用acme协议自动给我们重新颁发证书来续期
• spec.acme.server acme 协议的服务端，由官方给出
• spec.acme.privateKeySecretRef 指示此签发机构的私钥将要存储到哪个Secret对象中
• pec.acme.solvers.http01 指示签发机构使用HTTP-01的方式进行acme协议 (还可以用DNS方式，acme协议的目的是证明这台机器和域名都是属于你的，然后才准许给你颁发证书)

4、为域名创建certificate

这里通过一个我自己的域名blog.ssgeek.com来进行测试，此域名已经修改dns为公网地址

# cat certificate.yaml 
apiVersion: cert-manager.io/v1alpha2
kind: Certificate
metadata:
  name: blog
  namespace: default
spec:
  secretName: blog-tls
  issuerRef:
    name: letsencrypt-prod
    kind: ClusterIssuer
  duration: 2160h
  renewBefore: 360h
  keyEncoding: pkcs1
  dnsNames:
  - blog.ssgeek.com

说明：

• spec.secretName 指示证书最终存到哪个 Secret 中
• spec.issuerRef.kind 值为 ClusterIssuer 说明签发机构不在本 namespace 下，而是在全局
• spec.issuerRef.name 我们创建的签发机构的名称 (ClusterIssuer.metadata.name)
• spec.duration 证书过期时间
• spec.renewBefore 在过期前自动更新
• spec.dnsNames 指示该证书的可以用于哪些域名
• 更多选项可以参照官方文档

创建并检查相应资源

# kubectl apply -f certificate.yaml 
certificate.cert-manager.io/blog created
# kubectl get certificate
NAME   READY   SECRET     AGE
blog   True    blog-tls   36s
# kubectl get secrets |grep blog-tls
blog-tls               kubernetes.io/tls                     3      52s
# kubectl describe secrets blog-tls 
Name:         blog-tls
Namespace:    default
Labels:       <none>
Annotations:  cert-manager.io/alt-names: blog.ssgeek.com
              cert-manager.io/certificate-name: blog
              cert-manager.io/common-name: blog.ssgeek.com
              cert-manager.io/ip-sans: 
              cert-manager.io/issuer-kind: ClusterIssuer
              cert-manager.io/issuer-name: letsencrypt-prod
              cert-manager.io/uri-sans: 

Type:  kubernetes.io/tls

Data
====
ca.crt:   0 bytes
tls.crt:  3558 bytes
tls.key:  1675 bytes

在创建时查看cert-namager的日志

# kubectl -n cert-manager logs -f cert-manager-5cbcb9f4f5-4kks2
...
I0305 05:50:13.817322       1 controller.go:129] cert-manager/controller/certificates "msg"="syncing item" "key"="default/blog" 
I0305 05:50:14.317351       1 conditions.go:155] Setting lastTransitionTime for Certificate "blog" condition "Ready" to 2020-03-05 05:50:14.317341236 +0000 UTC m=+2213.785243238
I0305 05:50:14.525738       1 controller.go:135] cert-manager/controller/certificates "msg"="finished processing work item" "key"="default/blog" 
I0305 05:50:14.525812       1 controller.go:129] cert-manager/controller/certificates "msg"="syncing item" "key"="default/blog" 
I0305 05:50:14.526251       1 sync.go:367] cert-manager/controller/certificates "msg"="no existing CertificateRequest resource exists, creating new request..." "related_resource_kind"="Secret" "related_resource_name"="blog-tls" "related_resource_namespace"="default" "resource_kind"="Certificate" "resource_name"="blog" "resource_namespace"="default" 
I0305 05:50:14.774094       1 controller.go:129] cert-manager/controller/certificaterequests-issuer-ca "msg"="syncing item" "key"="default/blog-109727931" 
I0305 05:50:14.774118       1 controller.go:129] cert-manager/controller/certificaterequests-issuer-selfsigned "msg"="syncing item" "key"="default/blog-109727931" 
I0305 05:50:14.774135       1 sync.go:379] cert-manager/controller/certificates "msg"="created certificate request" "related_resource_kind"="Secret" "related_resource_name"="blog-tls" "related_resource_namespace"="default" "resource_kind"="Certificate" "resource_name"="blog" "resource_namespace"="default" "request_name"="blog-109727931"

如果创建出来的certificate状态为False，可以通过以下命令查看相关信息

# kubectl get challenge

如果有相应的challenge，通过kubectl describe检查，例如我这里之前创建失败时检查的错误信息如下

出现此问题的原因是我把此域名的解析设置为了内网地址，官方的颁发证书机构接口地址无法访问到，因此必须解析在公网，并保证服务暴露在公网

5、在ingress中引用对应的secret

生成的证书最终绑定在对应的域名服务下，这里我运行了一个nginx pod，创建了对应的service和ingress资源，在ingress资源中声明了此secret，由于部署了cert-maganer，在ingress中，还支持更多的注解，可以参考官方文档

yaml内容如下

apiVersion: apps/v1
kind: Deployment
metadata:
  name: nginx
  namespace: default
spec:
  replicas: 1
  selector:
    matchLabels:
      app: nginx
  template:
    metadata:
      labels:
        app: nginx
    spec:
      containers:
      - image: nginx:1.15 
        imagePullPolicy: IfNotPresent
        name: nginx

---
apiVersion: v1
kind: Service
metadata:
  name: nginx
  namespace: default
spec:
  selector:
    app: nginx
  ports:
  - name: nginx
    port: 80
    targetPort: 80

---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: nginx
  namespace: default
  annotations:
    kubernietes.io/ingress.class: "nginx"
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
spec:
  tls:
  - hosts:
    - blog.ssgeek.com
    secretName: blog-tls
  rules:
    - host: blog.ssgeek.com
      http:
        paths:
        - path: /
          backend:
            serviceName: nginx
            servicePort: 80

然后通过域名访问，检查证书是否正常

6、自动化颁发证书

上述内容是通过根据域名创建certificate最终得到的签名证书，再配置到ingress中使用，还不够自动化。没错，其实官方给出了自动通过ClusterIssuer颁发证书的做法，只需要在ingress中添加相应注解即可

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: nginx
  namespace: default
  annotations:
    kubernietes.io/ingress.class: "nginx"
    cert-manager.io/cluster-issuer: "letsencrypt-prod"
    kubernetes.io/tls-acme: "true"
spec:
  tls:
  - hosts:
    - blog.ssgeek.com
    secretName: blog-tls
  rules:
    - host: blog.ssgeek.com
      http:
        paths:
        - path: /
          backend:
            serviceName: nginx
            servicePort: 80

创建ingress资源，就会发现自动创建了certificate,得到secret
浏览器访问，出现307的http临时重定向到https，也可以继续添加一个注解强制进行强制重定向

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: nginx
  namespace: default
  annotations:
    kubernietes.io/ingress.class: "nginx"
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
    nginx.ingress.kubernetes.io/force-ssl-redirect: "true"
    cert-manager.io/cluster-issuer: "letsencrypt-prod"
    kubernetes.io/tls-acme: "true"
spec:
  tls:
  - hosts:
    - blog.ssgeek.com
    secretName: blog-tls
  rules:
    - host: blog.ssgeek.com
      http:
        paths:
        - path: /
          backend:
            serviceName: nginx
            servicePort: 80

参考：
官方文档
DIGITALOCEAN社区
Happiness"Blog