mac下安装tensorflow serving步骤及踩的坑

参考：

https://weiminwang.blog/2017/09/12/introductory-guide-to-tensorflow-serving/  
https://blog.csdn.net/bingningning/article/details/72933932
https://blog.csdn.net/wc781708249/article/details/78596459

1. clone tf_serving 资料库

git clone --recurse-submodules https://github.com/tensorflow/serving
cd serving$

2. 通过bazel 创建tensorflow_serving(初次耗时会比较长) （bazel没安装的可以https://pan.baidu.com/s/1pKLKnUz  下载，直接shell安装既可以）

bazel build tensorflow_serving/... 

3. 测试安装 

bazel test tensorflow_serving/...

这一步骤要注意python包里面必须含有grpc包，没有的话最后一步会报错,安装grpc包见如下   sudo pip install grpcio --ignore-installed six    ，这里有个问题要注意了一般mac下面有两个python环境，一个系统自带的2.x，另外一个可能是自己装的3，对应的pip 也可以安装两个，pip具体装法是在官网下载pip源码，sudo  python  setup.py install既可以，一般pip3对应的是python3，pip2对应的是python2

4.训练模型

import tensorflow as  tf
import  numpy as np
import  os
tf.app.flags.DEFINE_integer('training_iteration', 300,
'number of training iterations.')
tf.app.flags.DEFINE_integer('model_version', 1, 'version number of the model.')
tf.app.flags.DEFINE_string('work_dir', 'model/', 'Working directory.')
FLAGS = tf.app.flags.FLAGS

sess = tf.InteractiveSession()

x = tf.placeholder('float', shape=[None, 3])
y_ = tf.placeholder('float', shape=[None, 1])
w = tf.get_variable('w', shape=[3, 1], initializer=tf.truncated_normal_initializer)
b = tf.get_variable('b', shape=[1], initializer=tf.zeros_initializer)

sess.run(tf.global_variables_initializer())

y = tf.matmul(x, w) + b

ms_loss = tf.reduce_mean((y - y_) ** 2)

train_step = tf.train.GradientDescentOptimizer(0.005).minimize(ms_loss)

train_x = np.random.randn(1000, 3)
# let the model learn the equation of y = x1 * 1 + x2 * 2 + x3 * 3
train_y = np.sum(train_x * np.array([1, 2, 3]) + np.random.randn(1000, 3) / 100, axis=1).reshape(-1, 1)

train_loss = []

for _ in range(FLAGS.training_iteration):
    loss, _ = sess.run([ms_loss, train_step], feed_dict={x: train_x, y_: train_y})
    train_loss.append(loss)
print('Training error %g' % loss)
export_path_base = FLAGS.work_dir
export_path = os.path.join(
    tf.compat.as_bytes(export_path_base),
    tf.compat.as_bytes(str(FLAGS.model_version)))
print('Exporting trained model to', export_path)
builder = tf.saved_model.builder.SavedModelBuilder(export_path)

tensor_info_x = tf.saved_model.utils.build_tensor_info(x)
tensor_info_y = tf.saved_model.utils.build_tensor_info(y)

prediction_signature = (
    tf.saved_model.signature_def_utils.build_signature_def(
        inputs={'input': tensor_info_x},
        outputs={'output': tensor_info_y},
        method_name=tf.saved_model.signature_constants.PREDICT_METHOD_NAME))

legacy_init_op = tf.group(tf.tables_initializer(), name='legacy_init_op')

builder.add_meta_graph_and_variables(
    sess, [tf.saved_model.tag_constants.SERVING],
    signature_def_map={
        'prediction':
            prediction_signature,
    },
    legacy_init_op=legacy_init_op)

builder.save()

print('Done exporting!')

print('Done training!')

把1/*目录下面的文件copy  到/tmp/mode 下面   

 

5.启动服务

bazel-bin/tensorflow_serving/model_servers/tensorflow_model_server  --port=9000 --model_name=example_model --model_base_path=/tmp/model &> my_log &

6.写客户端调用，这里有点坑

写客户端调用必须依赖于tensorflow-serving-api,  但是该包在python2 下可以通过pip install 安装， 在python3下不行，可以参考官网说明https://www.tensorflow.org/serving/setup

先写client.py代码见如下：

from grpc.beta import implementations
import numpy
import tensorflow as tf
from tensorflow_serving.apis import predict_pb2
from tensorflow_serving.apis import prediction_service_pb2

tf.app.flags.DEFINE_string('server', 'localhost:9000', 'PredictionService host:port')
FLAGS = tf.app.flags.FLAGS


def do_inference(hostport):
    """Tests PredictionService with concurrent requests.
    Args:
    hostport: Host:port address of the Prediction Service.
    Returns:
    pred values, ground truth label
    """
    # create connection
    host, port = hostport.split(':')
    channel = implementations.insecure_channel(host, int(port))
    stub = prediction_service_pb2.beta_create_PredictionService_stub(channel)

    # initialize a request
    request = predict_pb2.PredictRequest()
    request.model_spec.name = 'example_model'
    request.model_spec.signature_name = 'prediction'

    # Randomly generate some test data
    temp_data = numpy.random.randn(10, 3).astype(numpy.float32)
    data, label = temp_data, numpy.sum(temp_data * numpy.array([1, 2, 3]).astype(numpy.float32), 1)
    request.inputs['input'].CopyFrom(
        tf.contrib.util.make_tensor_proto(data, shape=data.shape))

    # predict
    result = stub.Predict(request, 5.0)  # 5 seconds
    return result, label


def main(_):
    if not FLAGS.server:
        print('please specify server host:port')
        return

    result, label = do_inference(FLAGS.server)
    print('Result is: ', result)
    print('Actual label is: ', label)


if __name__ == '__main__':
    tf.app.run()

直接运行  python client.py --server localhost:9000

结果：

    float_val: 0.342217445374
    float_val: 1.61116909981
    float_val: 0.62158203125
    float_val: 0.404416233301
    float_val: 4.7025885582
    float_val: -3.33517575264
    float_val: 4.36096096039
    float_val: -0.214091524482
    float_val: 1.7283244133
    float_val: 1.58422541618
  }
}
model_spec {
  name: "example_model"
  version {
    value: 1
  }
  signature_name: "prediction"
}
)
('Actual label is: ', array([ 0.35279357,  1.5041277 ,  0.8823354 ,  0.39209065,  4.87715   ,
       -3.5436552 ,  4.7208014 , -0.44354367,  1.624176  ,  1.9318433 ],
      dtype=float32))

python3不能pip install解决方案要自己弄，幸好一些老外给出了解决方案，参考：

https://github.com/tensorflow/serving/issues/700

包下载链接:https://pypi.python.org/pypi/tensorflow-serving-api/1.4.0

包下载下来用unzip解压，把源码放到项目里面，python3运行那个客户端的代码如下:

/anaconda/bin/python3 /Users/shuubiasahi/Documents/python/credit-tftextclassify/prediction.py
WARNING:tensorflow:From /anaconda/lib/python3.5/site-packages/tensorflow/contrib/learn/python/learn/datasets/base.py:198: retry (from tensorflow.contrib.learn.python.learn.datasets.base) is deprecated and will be removed in a future version.
Instructions for updating:
Use the retry module or similar alternatives.
Result is:  outputs {
  key: "output"
  value {
    dtype: DT_FLOAT
    tensor_shape {
      dim {
        size: 10
      }
      dim {
        size: 1
      }
    }
    float_val: 1.8039723634719849
    float_val: -0.8360933661460876
    float_val: 1.624690055847168
    float_val: 1.5464890003204346
    float_val: -2.1499826908111572
    float_val: -3.8819611072540283
    float_val: -3.182767391204834
    float_val: -3.6825945377349854
    float_val: -0.13154874742031097
    float_val: -2.125609874725342
  }
}

Actual label is:  [ 2.0671349  -0.96695495  1.9177835   1.5574172  -2.199987   -4.2190337
 -3.4650352  -3.7537768  -0.19390547 -2.1524453 ]

也算是完美解决了，python3下不能直接pip  install  tensorflow-serving-api的问题

7一些总结

bazel安装要注意点，只要bazel安装好了接下来就很快，还有tensorflow-serving是比较占用空间，编译好之后至少要20g的缓存空间