windows部署tensorflow serving

将包含编译成功的tensorflow_model_serving.exe的文件拷到目标机器，如D:\TFServing目录下。

增加运行时库

由于该文件是在windows下使用vs2019的msvc编译器编译完成，需要对应的运行时库的支持。通过测试，目前需要用到的库文件主要有

vcruntime140_1.dll
vcruntime140.dll
msvcp140.dll

将运行时库放入库文件搜索路径，C:\Windows\System32下即可。

修改模型配置文件

修改配置文件的路径，注意路径分隔符使用双反斜杠\\或者单正斜杠/。如

model_config_list:{
	config:{	          			  
	name:"model_state",	
	base_path:"D:/TFServing/serving/tensorflow_serving/servables/tensorflow/testdata/multiModel/model_state",
	model_platform:"tensorflow"
    },    
    config:{		
    name:"model_ots",    
    base_path:"D:/TFServing/serving/tensorflow_serving/servables/tensorflow/testdata/multiModel/model_ots",
    model_platform:"tensorflow"
    }
}

仅适用于采用配置文件加载模型的情况。

命令行执行

通过win+R打开运行程序，输入cmd进入命令行窗口，进入文件所在目录，使用–model_base_path及–model_name分别指定单个模型的路径及模型名称，或使用–model_config_file指定配置文件（配置文件包含多个模型的路径及文件名），如

//切换磁盘
C:\Users\byzantine>D:
//进入指定目录
D:\>cd TFServing
//执行命令,使用配置文件
D:\TFServing>tensorflow_model_server.exe --modle_config_file=D:\TFServing\model\multiModel\model.config --rest_api_port=8501
//指定单个文件
//D:\TFServing>tensorflow_model_server.exe --model_base_path=D:\TFServing\saved_model_half_plus_two_cpu --model_name=half_plus_two --rest_api_port=8501

当屏幕上呈现如下输出表示服务已正常开启

...
2021-10-13 16:21:11.190345: I tensorflow_serving/core/loader_harness.cc:87] Successfully loaded servable version {name: half_plus_two version: 123}
2021-10-13 16:21:11.192614: I tensorflow_serving/model_servers/server_core.cc:486] Finished adding/updating models
2021-10-13 16:21:11.192739: I tensorflow_serving/model_servers/server.cc:133] Using InsecureServerCredentials
2021-10-13 16:21:11.192771: I tensorflow_serving/model_servers/server.cc:383] Profiler service is enabled
2021-10-13 16:21:11.195258: I tensorflow_serving/model_servers/server.cc:409] Running gRPC ModelServer at 0.0.0.0:8500 ...
[evhttp_server.cc : 249] NET_LOG: Entering the event loop ...
2021-10-13 16:21:11.197798: I tensorflow_serving/model_servers/server.cc:430] Exporting HTTP/REST API at:localhost:8501 ...

可通过客户端进行后续测试。