Kubernetes入门(四)——如何在Kubernetes中部署一个可对外服务的Tensorflow机器学习模型

机器学习模型常用Docker部署，而如何对Docker部署的模型进行管理呢？工业界的解决方案是使用Kubernetes来管理、编排容器。Kubernetes的理论知识不是本文讨论的重点，这里不再赘述，有关Kubernetes的优点读者可自行Google。笔者整理的Kubernetes入门系列的侧重点是如何实操，前三节介绍了Kubernets的安装、Dashboard的安装，以及如何在Kubernetes中部署一个无状态的应用，本节将讨论如何在Kubernetes中部署一个可对外服务的Tensorflow机器学习模型，作为Kubernetes入门系列的结尾。

希望Kubernetes入门系列能对K8S初学者提供一些参考，对文中描述有不同观点，或者对工业级部署与应用机器学习算法模型有什么建议，欢迎大家在评论区讨论与交流~~~

1. Docker中运行TensorFolw Serving

• 运行half_plus_two模型 ^[1]

# Download the TensorFlow Serving Docker image and repo
docker pull tensorflow/serving

mkdir /data0/modules
cd /data0/modules
git clone https://github.com/tensorflow/serving
# Location of demo models
TESTDATA="/data0/modules/serving/tensorflow_serving/servables/tensorflow/testdata/"

# Start TensorFlow Serving container and open the REST API port
docker run -dit --rm -p 8501:8501 
-v /data0/modules/serving/tensorflow_serving/servables/tensorflow/testdata/saved_model_half_plus_two_cpu:/models/half_plus_two 
-e MODEL_NAME=half_plus_two  tensorflow/serving 

# Query the model using the predict API
curl -d '{"instances": [1.0, 2.0, 5.0]}' 
    -X POST http://localhost:8501/v1/models/half_plus_two:predict

# Returns => { "predictions": [2.5, 3.0, 4.5] }

2. 构建TensorFolw模型的Docker镜像

• 后台运行serving容器

docker run -d --rm --name serving_base tensorflow/serving

• 拷贝模型数据到容器中的model目录

docker cp /data0/modules/serving/tensorflow_serving/servables/tensorflow/testdata/saved_model_half_plus_two_cpu serving_base:/models/half_plus_two

• 生成关于模型的镜像

docker commit --change "ENV MODEL_NAME half_plus_two" serving_base ljh/half_plus_two

• 停止serving容器

docker kill serving_base
docker rm serving_base

• 启动服务

docker run -dit --rm -p 8501:8501 
-e MODEL_NAME=half_plus_two  ljh/half_plus_two

• 查询模型

curl -d '{"instances": [1.0, 2.0, 5.0]}'    -X POST http://localhost:8501/v1/models/half_plus_two:predict

# Returns => { "predictions": [2.5, 3.0, 4.5] }

3. Kubernetes部署TensorFolw模型

创建关于模型的Deployment

• yaml文件

cat deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: halfplustwo-deployment
spec:
  selector:
    matchLabels:
      app: halfplustwo
  replicas: 1
  template:
    metadata:
      labels:
        app: halfplustwo
    spec:
      containers:
        - name: halfplustwo
          image: ljh/half_plus_two:latest
          imagePullPolicy: IfNotPresent
          ports:
            - containerPort: 8501
              name: restapi
            - containerPort: 8500
              name: grpc

• 创建一个Deployment:

kubectl apply -f deployment.yaml

• 展示Deployment相关信息:

kubectl get deployment -o wide
kubectl describe deployment halfplustwo-deployment

• 列出deployment创建的pods:

kubectl get pods -l app=halfplustwo

• 展示某一个pod信息

kubectl describe pod <pod-name>

使用service暴露你的应用

• yaml文件

cat service.yaml
apiVersion: v1
kind: Service
metadata:
  labels:
    run: halfplustwo-service
  name: halfplustwo-service
spec:
  ports:
    - port: 8501
      targetPort: 8501
      name: restapi
    - port: 8500
      targetPort: 8500
      name: grpc
  selector:
    app: halfplustwo
  type: LoadBalancer

• 启动service

kubectl create -f service.yaml
or
kubectl apply -f service.yaml

• 查看service

kubectl get service
#output:
NAME                  TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)                         AGE
halfplustwo-service   LoadBalancer   10.96.181.116   <pending>     8501:30771/TCP,8500:31542/TCP   4s
kubernetes            ClusterIP      10.96.0.1       <none>        443/TCP                         8d
nginx                 NodePort       10.96.153.10    <none>        80:30088/TCP                    29h

测试

curl -d '{"instances": [1.0, 2.0, 5.0]}'    -X POST http://localhost:8501/v1/models/half_plus_two:predict
{"predictions": [2.5, 3.0, 4.5]}

删除deployment和service

kubectl delete -f deployment.yaml
kubectl delete -f service.yaml

4. 参考资料

[1] https://www.tensorflow.org/tfx/serving/docker    TensorFlow Serving 与 Docker
[2] https://www.tensorflow.org/tfx/serving/serving_kubernetes?hl=zh_cn   将TensorFlow Serving与 Kubernetes结合使用
[3] https://towardsdatascience.com/scaling-machine-learning-models-using-tensorflow-serving-kubernetes-ed00d448c917  Scaling Machine Learning models using Tensorflow Serving & Kubernetes
[4] http://www.tuwee.cn/2019/03/03/Kubernetes+Tenserflow-serving%E6%90%AD%E5%BB%BA%E5%8F%AF%E5%AF%B9%E5%A4%96%E6%9C%8D%E5%8A%A1%E7%9A%84%E6%9C%BA%E5%99%A8%E5%AD%A6%E4%B9%A0%E5%BA%94%E7%94%A8/ Kubernetes+Tenserflow-serving搭建可对外服务的机器学习应用