zoukankan      html  css  js  c++  java
  • Kubernetes使用Keda进行弹性伸缩,更合理利用资源

    我最新最全的文章都在 南瓜慢说 www.pkslow.com ,欢迎大家来喝茶!

    1 简介

    Kubernetes自带的HPA是只支持CPU/MEM的,很多时候我们并不根据这两项指标来进行伸缩资源。比如消费者不断处理MQ的消息,我们希望MQ如果堆积过多,就启动更多的消费者来处理任务。而Keda给了我们很多选择。

    KEDA 是 Kubernetes 基于事件驱动的自动伸缩工具,通过 KEDA 我们可以根据需要处理的事件数量来驱动 Kubernetes 中任何容器的扩展。KEDA 可以直接部署到任何 Kubernetes 集群中和标准的组件一起工作。

    Keda所支持的事件源非常丰富,本文我们以RabbitMQ为例进行演示。

    2 安装Keda

    安装的方法很多,我们直接通过yaml文件来安装,这样还可以修改镜像地址等。先从( https://github.com/kedacore/keda/releases/download/v2.2.0/keda-2.2.0.yaml )下载yaml文件,然后执行:

    $ kubectl apply -f ~/Downloads/keda-2.2.0.yaml
    namespace/keda created
    customresourcedefinition.apiextensions.k8s.io/clustertriggerauthentications.keda.sh created
    customresourcedefinition.apiextensions.k8s.io/scaledjobs.keda.sh created
    customresourcedefinition.apiextensions.k8s.io/scaledobjects.keda.sh created
    customresourcedefinition.apiextensions.k8s.io/triggerauthentications.keda.sh created
    serviceaccount/keda-operator created
    clusterrole.rbac.authorization.k8s.io/keda-external-metrics-reader created
    clusterrole.rbac.authorization.k8s.io/keda-operator created
    rolebinding.rbac.authorization.k8s.io/keda-auth-reader created
    clusterrolebinding.rbac.authorization.k8s.io/keda-hpa-controller-external-metrics created
    clusterrolebinding.rbac.authorization.k8s.io/keda-operator created
    clusterrolebinding.rbac.authorization.k8s.io/keda:system:auth-delegator created
    service/keda-metrics-apiserver created
    deployment.apps/keda-metrics-apiserver created
    deployment.apps/keda-operator created
    apiservice.apiregistration.k8s.io/v1beta1.external.metrics.k8s.io created
    

    检查一下是否都已经启动完成:

    $ kubectl get all -n keda
    NAME                                          READY   STATUS    RESTARTS   AGE
    pod/keda-metrics-apiserver-55dc9f9498-smc2d   1/1     Running   0          2m41s
    pod/keda-operator-59dcf989d6-pxcbb            1/1     Running   0          2m41s
    
    NAME                             TYPE        CLUSTER-IP      EXTERNAL-IP   PORT(S)          AGE
    service/keda-metrics-apiserver   ClusterIP   10.104.255.44   <none>        443/TCP,80/TCP   2m41s
    
    NAME                                     READY   UP-TO-DATE   AVAILABLE   AGE
    deployment.apps/keda-metrics-apiserver   1/1     1            1           2m42s
    deployment.apps/keda-operator            1/1     1            1           2m42s
    
    NAME                                                DESIRED   CURRENT   READY   AGE
    replicaset.apps/keda-metrics-apiserver-55dc9f9498   1         1         1       2m42s
    replicaset.apps/keda-operator-59dcf989d6            1         1         1       2m42s
    

    也可以看到镜像多了:

    $ docker images | grep keda
    ghcr.io/kedacore/keda-metrics-apiserver                 2.2.0                                            a43d40453368        6 weeks ago         95.3MB
    ghcr.io/kedacore/keda                                   2.2.0                                            42b88f042914        6 weeks ago         83MB
    

    如果要卸载请执行:

    $ kubectl delete -f ~/Downloads/keda-2.2.0.yaml
    

    3 安装RabbitMQ

    为了快速安装,也方便日后删除,我们通过Helm来安装RabbitMQ。

    查看可用的chart:

    $ helm search repo rabbit
    

    执行安装:

    $ helm install azure-rabbitmq azure/rabbitmq
    

    检查一下:

    $ helm list
    NAME          	NAMESPACE	REVISION	UPDATED                             	STATUS  	CHART               	APP VERSION
    azure-ingress 	default  	1       	2021-02-14 01:21:07.212107 +0800 CST	deployed	nginx-ingress-1.41.3	v0.34.1    
    azure-rabbitmq	default  	1       	2021-05-05 11:29:06.979437 +0800 CST	deployed	rabbitmq-6.18.2     	3.8.2
    

    用户名为user,密码获取如下:

    $ echo "Password      : $(kubectl get secret --namespace default azure-rabbitmq -o jsonpath="{.data.rabbitmq-password}" | base64 --decode)"
    Password      : YNsEayx8w2
    

    4 测试

    部署消费者,注意这里有个MQ连接信息和加密,要根据自己情况修改。

    $ kubectl apply -f src/main/kubernetes/deploy-consumer.yaml
    secret/rabbitmq-consumer-secret created
    deployment.apps/rabbitmq-consumer created
    scaledobject.keda.sh/rabbitmq-consumer created
    triggerauthentication.keda.sh/rabbitmq-consumer-trigger created
    

    查看deployment,发现是没有Pod创建,因为还不需要处理,MQ现在的队列为0。

    $ kubectl get deployments
    NAME                                          READY   UP-TO-DATE   AVAILABLE   AGE
    azure-ingress-nginx-ingress-controller        1/1     1            1           80d
    azure-ingress-nginx-ingress-default-backend   1/1     1            1           80d
    rabbitmq-consumer                             0/0     0            0           131m
    

    部署生产者,往MQ发送消息:

    $ kubectl apply -f src/main/kubernetes/deploy-publisher-job.yaml
    job.batch/rabbitmq-publish created
    

    可以看到,慢慢消费者就起来了,并且创建了越来越多的Pod来处理MQ:

    $ kubectl get deployments rabbitmq-consumer
    NAME                READY   UP-TO-DATE   AVAILABLE   AGE
    rabbitmq-consumer   1/1     1            1           167m
    
    $ kubectl get deployments rabbitmq-consumer
    NAME                READY   UP-TO-DATE   AVAILABLE   AGE
    rabbitmq-consumer   3/4     4            3           168m
    
    $ kubectl get deployments rabbitmq-consumer
    NAME                READY   UP-TO-DATE   AVAILABLE   AGE
    rabbitmq-consumer   4/8     8            4           168m
    
    $ kubectl get deployments rabbitmq-consumer
    NAME                READY   UP-TO-DATE   AVAILABLE   AGE
    rabbitmq-consumer   6/8     8            6           169m
    $ kubectl get deployments rabbitmq-consumer
    NAME                READY   UP-TO-DATE   AVAILABLE   AGE
    rabbitmq-consumer   0/0     0            0           171m
    

    查看Deployment的Event也可以看到结果:

    Events:
      Type    Reason             Age                   From                   Message
      ----    ------             ----                  ----                   -------
      Normal  ScalingReplicaSet  5m55s (x2 over 172m)  deployment-controller  Scaled up replica set rabbitmq-consumer-7b477f78b4 to 1
      Normal  ScalingReplicaSet  5m6s                  deployment-controller  Scaled up replica set rabbitmq-consumer-7b477f78b4 to 4
      Normal  ScalingReplicaSet  4m6s                  deployment-controller  Scaled up replica set rabbitmq-consumer-7b477f78b4 to 8
      Normal  ScalingReplicaSet  3m5s                  deployment-controller  Scaled up replica set rabbitmq-consumer-7b477f78b4 to 16
      Normal  ScalingReplicaSet  3m3s (x2 over 172m)   deployment-controller  Scaled down replica set rabbitmq-consumer-7b477f78b4 to 0
    

    处理完成后,又会回到0了。

    总结

    代码请查看:https://github.com/LarryDpk/pkslow-samples


    欢迎关注微信公众号<南瓜慢说>,将持续为你更新...

    多读书,多分享;多写作,多整理。

  • 相关阅读:
    PAT (Advanced Level) 1086. Tree Traversals Again (25)
    PAT (Advanced Level) 1085. Perfect Sequence (25)
    PAT (Advanced Level) 1084. Broken Keyboard (20)
    PAT (Advanced Level) 1083. List Grades (25)
    PAT (Advanced Level) 1082. Read Number in Chinese (25)
    HDU 4513 吉哥系列故事――完美队形II
    POJ Oulipo KMP 模板题
    POJ 3376 Finding Palindromes
    扩展KMP
    HDU 2289 Cup
  • 原文地址:https://www.cnblogs.com/larrydpk/p/14953946.html
Copyright © 2011-2022 走看看