[tf_serving] 在docker中启动tensorflow_serving

首先在PC的命令行下将model的.pb文件copy到容器中：

cd <path to serving model files>
sudo docker cp ./serving_am tf_container_asr:/serving

 然后进入容器的命令行启动相关服务：

root@2effa7569ce3:/serving# tensorflow_model_server --port=9000 --model_name=ASR_am1 --model_base_path=/serving/serving_am   --saved_model_tags="serve,ASR_am" &> vc2.1_log &
[1] 712
root@2effa7569ce3:/serving# ls
export_model  serving  vc2.1_log  vc_log_0118  vc_test_log_0118
root@2effa7569ce3:/serving# cat vc2.1_log 
2019-03-11 13:35:22.860622: I tensorflow_serving/model_servers/server.cc:82] Building single TensorFlow model file config:  model_name: VC2.1 model_base_path: /serving/export_model
2019-03-11 13:35:22.860848: I tensorflow_serving/model_servers/server_core.cc:461] Adding/updating models.
2019-03-11 13:35:22.860871: I tensorflow_serving/model_servers/server_core.cc:558]  (Re-)adding model: VC2.1
2019-03-11 13:35:22.961789: I tensorflow_serving/core/basic_manager.cc:739] Successfully reserved resources to load servable {name: VC2.1 version: 106}
2019-03-11 13:35:22.961846: I tensorflow_serving/core/loader_harness.cc:66] Approving load for servable version {name: VC2.1 version: 106}
2019-03-11 13:35:22.961871: I tensorflow_serving/core/loader_harness.cc:74] Loading servable version {name: VC2.1 version: 106}
2019-03-11 13:35:22.961907: I external/org_tensorflow/tensorflow/contrib/session_bundle/bundle_shim.cc:363] Attempting to load native SavedModelBundle in bundle-shim from: /serving/export_model/00000106
2019-03-11 13:35:22.961924: I external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:31] Reading SavedModel from: /serving/export_model/00000106
2019-03-11 13:35:22.995097: I external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:54] Reading meta graph with tags { serve }
2019-03-11 13:35:23.135058: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:182] Restoring SavedModel bundle.
2019-03-11 13:35:23.423978: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:285] SavedModel load for tags { serve }; Status: success. Took 462027 microseconds.
2019-03-11 13:35:23.424085: I tensorflow_serving/servables/tensorflow/saved_model_warmup.cc:83] No warmup data file found at /serving/export_model/00000106/assets.extra/tf_serving_warmup_requests
2019-03-11 13:35:23.424713: I tensorflow_serving/core/loader_harness.cc:86] Successfully loaded servable version {name: VC2.1 version: 106}
2019-03-11 13:35:23.442710: I tensorflow_serving/model_servers/server.cc:286] Running gRPC ModelServer at 0.0.0.0:9000 ...

tensorflow_model_server  --port=9003 --model_name=VC2.1 --model_base_path=/serving/export_model --saved_model_tag="serve" &> vc2.1_log &
tensorflow_model_server  --port=9004 --model_name=VC2.2 --model_base_path=/serving/export_model2 --saved_model_tag="serve" &> vc2.2_log &

tensorflow_model_server --port=9001 --model_name=ASR_am --model_base_path=/serving/serving_am   --saved_model_tags="serve" &> am_log &
tensorflow_model_server --port=9002 --model_name=ASR_lm --model_base_path=/serving/serving_lm   --saved_model_tags="serve" &> lm_log &

 

注意必须要有有正确的tag，原因， 原因

可以通过saved_model_cli查看tag信息：

[currycode@mjrc-server11 log]$ saved_model_cli show --dir serving_am/00000330/ --all

MetaGraphDef with tag-set: 'serve, ASR_am' contains the following SignatureDefs:

signature_def['predict_Spec2Pinyin']:
  The given SavedModel SignatureDef contains the following input(s):
    inputs['spec'] tensor_info:
        dtype: DT_FLOAT
        shape: (-1, -1, 200, 1)
        name: the_inputs:0
  The given SavedModel SignatureDef contains the following output(s):
    outputs['score'] tensor_info:
        dtype: DT_FLOAT
        shape: (-1, -1, 230)
        name: dense_2/truediv:0
  Method name is: tensorflow/serving/predict