tensorflow serving部署概要

1. 目录结构

[@hbhly_SG21_41_53 pb_models]# tree textcnn/
textcnn/
└── 000
    ├── saved_model.pb
    └── variables
        ├── variables.data-00000-of-00001
        └── variables.index

2. 检查结果pb文件的default配置

saved_model_cli show --dir /root/huxiang/serving/pb_models/textcnn/000 --all

MetaGraphDef with tag-set: 'serve' contains the following SignatureDefs:

signature_def['serving_default']:
  The given SavedModel SignatureDef contains the following input(s):
    inputs['voice'] tensor_info:
        dtype: DT_FLOAT
        shape: (-1, 50)
        name: inputs:0
  The given SavedModel SignatureDef contains the following output(s):
    outputs['scores'] tensor_info:
        dtype: DT_FLOAT
        shape: (-1, 2)
        name: FC2/Softmax:0
  Method name is: tensorflow/serving/predict

3. 拉取镜像

docker pull tensorflow/serving:1.12.0

4. 部署

docker run -p 8501:8501 -p 8500:8500 -v "/root/huxiang/serving/pb_models/textcnn:/models/textcnn" -e MODEL_NAME=textcnn -e MODEL_BASE_PATH=/models/ -t tensorflow/serving:1.12.0 > d.log &

5. 测试

curl -d '{"inputs": [[1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1]]}' -X POST http://localhost:8501/v1/models/textcnn:predict  

6. 返回结果

{
    "outputs": [
        [
            0.098275,
            0.901725
        ]
    ]
}

7. python测试

import json
import numpy as np
import requests
from contextlib import contextmanager
from time import time

@contextmanager
def timer(name):
    t0 = time()
    yield
    # print(f'[{name}] done in {time() - t0:.2f} s')
    print(name + 'cost:' + str(time() - t0) + 's')

def test():
    data = json.dumps({
        "inputs": [[1] * 50]
    })
    headers = {"content-type": "application/json"}
    json_response = requests.post(
        'http://localhost:8501/v1/models/textcnn:predict',
        data=data, headers=headers)
    predictions = np.array(json.loads(json_response.text)['outputs'])
    # print(np.argmax(predictions, axis=-1))


if __name__ == '__main__':

    with timer('single...'):
        test()

    with timer('test...'):
        for i in range(1000):
            test()

    with timer('single...'):
        test()



"""
# 好像刚开始的时候速度有点慢
single...cost:0.0194900035858s
test...cost:2.31740784645s
single...cost:0.00248503684998s
    
# mac的速度
single...cost:0.016093969345092773s
test...cost:4.734021186828613s
single...cost:0.004168033599853516s
"""