zoukankan      html  css  js  c++  java
  • Deploying Keras model on Tensorflow Serving--

    keras训练了个二分类的模型。需求是把keras模型跑到 tensorflow serving上 (TensorFlow Serving 系统用于在生产环境中运行模型)

    keras模型转 tensorflow模型

    我把 keras模型转tensorflow serving模型所使用的方法如下:

    1、要拿到算法训练好的keras模型文件(一个HDF5文件)

    该文件应该包含:

    • 模型的结构,以便重构该模型
    • 模型的权重
    • 训练配置(损失函数,优化器等)
    • 优化器的状态,以便于从上次训练中断的地方开始

    2、编写 keras模型转tensorflow serving模型的代码

    import pandas as pd
    import os
    import tensorflow as tf
    
    tf.logging.set_verbosity(tf.logging.INFO)
    ...
    def build_model():
        ############
        ...
        return model
    
    
    def save_model_for_production(model, version, path='prod_models'):
        tf.keras.backend.set_learning_phase(1)
        if not os.path.exists(path):
            os.mkdir(path)
        export_path = os.path.join(
            tf.compat.as_bytes(path),
            tf.compat.as_bytes(version))
        builder = tf.saved_model.builder.SavedModelBuilder(export_path)
    
        model_input = tf.saved_model.utils.build_tensor_info(model.input)
        model_output = tf.saved_model.utils.build_tensor_info(model.output)
    
        prediction_signature = (
            tf.saved_model.signature_def_utils.build_signature_def(
                inputs={'inputs': model_input},
                outputs={'output': model_output},
                method_name=tf.saved_model.signature_constants.PREDICT_METHOD_NAME))
    
        with tf.keras.backend.get_session() as sess:
            builder.add_meta_graph_and_variables(
                sess=sess, tags=[tf.saved_model.tag_constants.SERVING],
                signature_def_map={
                    'predict':
                        prediction_signature,
                })
    
            builder.save()
    
    
    if __name__ == '__main__':
        model_file = './my_model.h5'
        if (os.path.isfile(model_file)):
            print('model file detected. Loading.')
            model = tf.keras.models.load_model(model_file)
        else:
            print('No model file detected.  Starting from scratch.')
            model = build_model()
            model.compile(loss='binary_crossentropy', optimizer="adam", metrics=['accuracy'])
            model.save(model_file)
    
        model.fit(X_train, y_train, batch_size=100, epochs=1, validation_data=(X_test, y_test))
        model.summary()
    
        export_path = "tf-model"
        save_model_for_production(model, "1", export_path)
    

    上面的例子将模型保存到 tf-model目录下
    tf-model目录结构如下:

    tf-model/
    └── 1
        ├── saved_model.pb
        └── variables
            ├── variables.data-00000-of-00001
            └── variables.index
    

    saved_model.pb 是能在 tensorflow serving跑起来的模型。

    3、跑模型

    tensorflow_model_server --port=9000 --model_name="username" --model_base_path="/data/models/tf-model/"
    

    标准输出如下(算法模型已成功跑起来了):

    Running ModelServer at 0.0.0.0:00 ...
    

    4、客户端代码

    #!/usr/bin/env python  
    # encoding: utf-8  
    
    """ 
    @version: v1.0 
    @author: zwqjoy 
    @contact: zwqjoy@163.com 
    @site: https://blog.csdn.net/zwqjoy 
    @file: client
    @time: 2018/6/29 15:02 
    """
    
    from __future__ import print_function
    from grpc.beta import implementations
    import tensorflow as tf
    
    from tensorflow_serving.apis import predict_pb2
    from tensorflow_serving.apis import prediction_service_pb2
    import numpy as np
    
    tf.app.flags.DEFINE_string('server', 'localhost:9000',
                               'PredictionService host:port')
    FLAGS = tf.app.flags.FLAGS
    
    
    def main(_):
        host, port = FLAGS.server.split(':')
        channel = implementations.insecure_channel(host, int(port))
        stub = prediction_service_pb2.beta_create_PredictionService_stub(channel)
        # Send request
    
        # See prediction_service.proto for gRPC request/response details.
        data = np.array([4, 0, 0, 0, 1, 0, 1])
        data = data.astype(np.float32)
    
        request = predict_pb2.PredictRequest()
        request.model_spec.name = 'username'  # 这个name跟tensorflow_model_server  --model_name="username" 对应
        request.model_spec.signature_name = 'predict'  # 这个signature_name  跟signature_def_map 对应
        request.inputs['inputs'].CopyFrom(
            tf.contrib.util.make_tensor_proto(data, shape=(1, 7)))  # shape跟 keras的model.input类型对应
        result = stub.Predict(request, 10.0)  # 10 secs timeout
        print(result)
    
    
    if __name__ == '__main__':
        tf.app.run()

    客户端跑出的结果是:

    outputs {
      key: "output"
      value {
        dtype: DT_FLOAT
        tensor_shape {
          dim {
            size: 1
          }
          dim {
            size: 1
          }
        }
        float_val: 0.976889811523
      }
    }
    

    float_val: 0.976889811523 就是我们需要的结果(概率)


    keras模型转 tensorflow模型的一些说明

    1、 keras 保存模型

    可以使用model.save(filepath)将Keras模型和权重保存在一个HDF5文件中,该文件将包含:

    • 模型的结构,以便重构该模型
    • 模型的权重
    • 训练配置(损失函数,优化器等)
    • 优化器的状态,以便于从上次训练中断的地方开始

    当然这个 HDF5 也可以是用下面的代码生成

    from keras.models import load_model
    model.save('my_model.h5')

    2、 keras 加载模型

    keras 加载模型(中间部分代码省略了):


    import numpy as np
    from keras.datasets import mnist
    from keras.utils import np_utils
    from keras.models import Sequential
    from keras.layers import Dense
    from keras.optimizers import SGD
    from keras.models import load_model
    # 载入数据
    (x_train,y_train),(x_test,y_test) = mnist.load_data()
    # (60000,28,28)
    print('x_shape:',x_train.shape)
    # (60000)
    print('y_shape:',y_train.shape)
    # (60000,28,28)->(60000,784)
    x_train = x_train.reshape(x_train.shape[0],-1)/255.0
    x_test = x_test.reshape(x_test.shape[0],-1)/255.0
    # 换one hot格式
    y_train = np_utils.to_categorical(y_train,num_classes=10)
    y_test = np_utils.to_categorical(y_test,num_classes=10)
    
    # 载入模型
    model = load_model('model.h5')
    
    # 评估模型
    loss,accuracy = model.evaluate(x_test,y_test)
    
    print('
    test loss',loss)
    print('accuracy',accuracy)
    
    # 训练模型
    model.fit(x_train,y_train,batch_size=64,epochs=2)
    
    # 评估模型
    loss,accuracy = model.evaluate(x_test,y_test)
    
    print('
    test loss',loss)
    print('accuracy',accuracy)
    
    # 保存参数,载入参数
    model.save_weights('my_model_weights.h5')
    model.load_weights('my_model_weights.h5')

    keras 模型转tensorflow serving 模型的一些坑

    希望能让新手少走一些弯路

    坑1:过时的生成方法

    有些方法已经过时了(例如下面这种):

    from tensorflow_serving.session_bundle import exporter
    
    export_path = ... # where to save the exported graph
    export_version = ... # version number (integer)
    
    saver = tf.train.Saver(sharded=True)
    model_exporter = exporter.Exporter(saver)
    signature = exporter.classification_signature(input_tensor=model.input,
                                                  scores_tensor=model.output)
    model_exporter.init(sess.graph.as_graph_def(),
                        default_graph_signature=signature)
    model_exporter.export(export_path, tf.constant(export_version), sess)
    

    如果使用这种过时的方法,用tensorflow serving 跑模型的时候会提示:

    WARNING:tensorflow:From test.py:107: Exporter.export (from tensorflow.contrib.session_bundle.exporter) is deprecated and will be removed after 2017-06-30.
    Instructions for updating:
    No longer supported. Switch to SavedModel immediately.
    

    从warning中 显然可以知道这种方法要被抛弃了,不再支持这种方法了, 建议我们转用 SaveModel方法。

    填坑大法: 使用 SaveModel

    def save_model_for_production(model, version, path='prod_models'):
        tf.keras.backend.set_learning_phase(1)
        if not os.path.exists(path):
            os.mkdir(path)
        export_path = os.path.join(
            tf.compat.as_bytes(path),
            tf.compat.as_bytes(version))
        builder = tf.saved_model.builder.SavedModelBuilder(export_path)
    
        model_input = tf.saved_model.utils.build_tensor_info(model.input)
        model_output = tf.saved_model.utils.build_tensor_info(model.output)
    
        prediction_signature = (
            tf.saved_model.signature_def_utils.build_signature_def(
                inputs={'inputs': model_input},
                outputs={'output': model_output},
                method_name=tf.saved_model.signature_constants.PREDICT_METHOD_NAME))
    
        with tf.keras.backend.get_session() as sess:
            builder.add_meta_graph_and_variables(
                sess=sess, tags=[tf.saved_model.tag_constants.SERVING],
                signature_def_map={
                    'predict':
                        prediction_signature,
                })
    
            builder.save()
    参考:
    https://www.jianshu.com/p/91aae37f1da6 
    Deploying Keras model on Tensorflow Serving with GPU support

    https://github.com/amir-abdi/keras_to_tensorflow

  • 相关阅读:
    POJ 1426 Find The Multiple(数论——中国同余定理)
    POJ 2253 Frogger(Dijkstra变形——最短路径最大权值)
    POJ 3790 最短路径问题(Dijkstra变形——最短路径双重最小权值)
    POJ 3278 Catch That Cow(模板——BFS)
    HDU 1071 The area
    HDU 1213 How Many Tables(模板——并查集)
    POJ 1611 The Suspects
    light oj 1214 Large Division
    POJ 1258 Agri-Net(Prim算法求解MST)
    POJ 2387 Til the Cows Come Home(模板——Dijkstra算法)
  • 原文地址:https://www.cnblogs.com/WayneZeng/p/9290691.html
Copyright © 2011-2022 走看看