【kubeflow】kf-serving一些记载

tensorflow

启动脚本:

apiVersion: "serving.kserve.io/v1beta1"
kind: "InferenceService"
metadata:
  labels:
    predict-phantom.amh-group.com/modelName: "lgb-test"
    predict-phantom.amh-group.com/modelVersion: "v1.2"
  name: "t"
spec:
  predictor:
    maxReplicas: 1
    minReplicas: 1
    canaryTrafficPercent: 100
    tensorflow:
      resources:
        limits:
          memory: "4Gi"
          cpu: "8"
        requests:
          memory: "4Gi"
          cpu: "8"
      runtimeVersion: 2.3.0
      storageUri: "http://10.13.68.11/nfs/t-model.zip"

模型信息接口:

跟tensorflow-serving一致，地址为http://kf01:30000/v1/models/t/metadata

host:t.default.amh-group.com

预测接口:

http://kf01:30000/v1/models/t:predict

参数:

{
    "inputs":{
        "x_k1":[[0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 405, 405, 405, 430, 430, 430, 430, 430, 430, 430, 430, 430, 430, 462, 462, 462, 462, 462, 462, 462, 462, 462, 462, 679, 679, 679, 679, 679]],
        "x_k2":[[75, 76, 74, 75, 0, 0, 1, 0, 0, 171, 188, 261, 2, 405, 405, 33, 430, 430, 679, 679, 679]],
        "x_k3":[[0, 0, 405, 430, 462, 679, 679, 679]],
        "x_v1":[[1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 1.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0]],
        "x_v2":[[100.0999984741211, 4.199999809265137, 0, 0]],
        "x_v3":[[0.0, 0.0, 0.0, 0.0, -1.0]],
        "is_training":false
    }
}

模型格式:

- 1

　　- saved_model.pb

　　- variables

pytorch

启动脚本：

apiVersion: serving.kserve.io/v1beta1
kind: InferenceService
metadata:
  name: "torchserve"
spec:
  predictor:
    pytorch:
      storageUri: gs://kfserving-examples/models/torchserve/image_classifier

模型预测/解释：

http://10.13.68.12:30000/v1/models/mnist:predict

http://10.13.68.12:30000/v1/models/mnist:explain

参数:

{
  "instances": [
    {
      "data": "iVBORw0KGgoAAAANSUhEUgAAABwAAAAcCAAAAABXZoBIAAAAw0lEQVR4nGNgGFggVVj4/y8Q2GOR83n+58/fP0DwcSqmpNN7oOTJw6f+/H2pjUU2JCSEk0EWqN0cl828e/FIxvz9/9cCh1zS5z9/G9mwyzl/+PNnKQ45nyNAr9ThMHQ/UG4tDofuB4bQIhz6fIBenMWJQ+7Vn7+zeLCbKXv6z59NOPQVgsIcW4QA9YFi6wNQLrKwsBebW/68DJ388Nun5XFocrqvIFH59+XhBAxThTfeB0r+vP/QHbuDCgr2JmOXoSsAAKK7bU3vISS4AAAAAElFTkSuQmCC",
      "target": 0
    }
  ]
}

参考文档https://github.com/kserve/kserve/tree/master/docs/samples/v1beta1/torchserve

模型格式:

├── config
│   ├── config.properties
├── model-store
│   ├── densenet_161.mar
│   ├── mnist.mar

需要参考torchserve的模型导出，较复杂【TODO】

PMML

启动脚本

apiVersion: "serving.kserve.io/v1beta1"
kind: "InferenceService"
metadata:
  name: "pmml-demo"
spec:
  predictor:
    pmml:
      storageUri: http://10.13.68.11/nfs/model.pmml

模型预测:

http://10.13.68.12:30000/v1/models/pmml-demo:predict

Host:pmml-demo.default.amh-group.com

请求参数:

{
    "predictions": [
        {
            "Species": "setosa",
            "Probability_setosa": 1.0,
            "Probability_versicolor": 0.0,
            "Probability_virginica": 0.0,
            "Node_Id": "2"
        }
    ]
}

自定义模型

apiVersion: serving.kserve.io/v1beta1
kind: InferenceService
metadata:
  name: embedding-test
spec:
  predictor:
    containers:
      - name: orochi
        image: 10.13.68.15:5000/orochi/embedding:v1

需要注意的是，这个容器启动必须是一个rest服务

 

修改域名

https://knative.dev/docs/serving/using-a-custom-domain/