【ubuntu】tensorflow tfserving 多模型

模型文件目录结构

multiModel
	pred0
		001
	pred1
		001
	pred2
		001

编辑配置文件

在saved_model目录下建立models.config

model_config_list:{
    config:{
      name:"model0",
      base_path:"/models/saved_model/pred0",
      model_platform:"tensorflow"
    },
    config:{
      name:"model1",
      base_path:"/models/saved_model/pred1",
      model_platform:"tensorflow"
    },
    config:{
      name:"model2",
      base_path:"/models/saved_model/pred2",
      model_platform:"tensorflow"
    } 
    config:{
      name:"model3",
      base_path:"/models/saved_model/pred3",
      model_platform:"tensorflow"
    },
    config:{
      name:"model4",
      base_path:"/models/saved_model/pred4",
      model_platform:"tensorflow"
    },
    config:{
      name:"model5",
      base_path:"/models/saved_model/pred5",
      model_platform:"tensorflow"
    },
    config:{
      name:"model6",
      base_path:"/models/saved_model/pred6",
      model_platform:"tensorflow"
    },
    config:{
      name:"model7",
      base_path:"/models/saved_model/pred7",
      model_platform:"tensorflow"
    },
    config:{
      name:"model8",
      base_path:"/models/saved_model/pred8",
      model_platform:"tensorflow"
    },
    config:{
      name:"model9",
      base_path:"/models/saved_model/pred9",
      model_platform:"tensorflow"
    },
}

运行tfserving容器

docker run -p 8501:8501 --mount type=bind,source=/usr/bigCong/saved_model/,target=/models/saved_model \
 -t tensorflow/serving --model_config_file=/models/saved_model/models.config

预测

import requests 
import numpy as np 
SERVER_URL = 'http://localhost:8501/v1/models/model3:predict'  
#注意SERVER_URL中的‘model3’是config文件中定义的模型name,不是文件夹名称

def prediction(): 
    predict_request='{"instances":%s}' % str([[[10]*7]*7]) 
    print(predict_request) 
    response = requests.post(SERVER_URL, data=predict_request) 
    print(response)
    prediction = response.json()['predictions'][0] 
    print(prediction) 

if __name__ == "__main__": 
    prediction()