【创建tensorflow severing容器出错】failed: Out of range: Read less bytes than requested【已解决】

〇、前言

最近在做tensorflow severing模型部署，拿了个yolov5训练出来的pt模型做实验。前几天一切正常，就在今天创建tensorflow severing容器的时候遇到了错误，环境、命令、模型、配置文件这些都没有变动，突然就出现了这个错误，发现网上遇到类似情况的极其稀少，找了很久在https://github.com/tensorflow/tensorflow/issues/21544中才找到了问题所在，并完美解决问题。如果遇到同样问题的同学看到了这篇文章，希望能帮到你！

一、问题描述

创建tensorflow severing容器的时候遇到了错误，环境、命令、模型、配置文件这些都没有变动，突然就出现了这个错误。

1. 错误摘要(f方便被搜索到)：

• E tensorflow_serving/util/retrier.cc:37] Loading servable: {name: yolov5_saved_model version: 1} failed: Out of range: Read less bytes than requested
• Failed to start server. Error: Unknown: 1 servable(s) did not become available: {{{name: yolov5_saved_model version: 1} due to error: Out of range: Read less bytes than requested}, }
• 2022-07-30 11:15:09.097717: I tensorflow_serving/core/basic_manager.cc:279] Unload all remaining servables in the manager.

2. 错误截图：

3. 完整错误代码如下：

（见附录）

二、问题分析：

错误是由文件损坏导致，具体怎么损坏，我想到两种可能：上传文件中断或出错、完好的文件由于磁盘问题导致损坏。

我的是第二种

参考了https://github.com/tensorflow/tensorflow/issues/21544才发现的。👇

三、解决方法：

删除模型及配置文件，重新上传。

如果没有提前没有备份，或者不知道文件是否损坏，则可以重新进行模型转换生成saved model模型，并新建配置文件。

下图是我重新上传的文件结构和文件截图。

models
|----model1
|---- ----1
|---- ---- ----assets
|---- ---- ---- ----variables
|---- ---- ---- ----variables.data-00000-of-00001
|---- ---- ----variables.index
|---- ---- ----saved_model.pb
|----model.config

参考

相似的问题：
https://github.com/tensorflow/tensorflow/issues/21544
https://bytemeta.vip/repo/Breta01/handwriting-ocr/issues/104

附录

ubuntu% docker run --rm -p 8500:8500 --mount type=bind,source=/media/userdata/zhangxw/TFSever_Test2/yolo_cow/models,target=/models/models -t tensorflow/serving:latest-gpu --model_config_file=/models/models/model.config --allow_version_labels_for_unavailable_models=true &
[1] 8565
ubuntu% 2022-07-30 11:10:08.016885: I external/org_tensorflow/tensorflow/stream_executor/platform/default/dso_loader.cc:53] Successfully opened dynamic library libcudart.so.11.0
2022-07-30 11:10:08.052728: I tensorflow_serving/model_servers/server_core.cc:465] Adding/updating models.
2022-07-30 11:10:08.052746: I tensorflow_serving/model_servers/server_core.cc:591]  (Re-)adding model: yolov5_saved_model
2022-07-30 11:10:08.153403: I tensorflow_serving/core/basic_manager.cc:740] Successfully reserved resources to load servable {name: yolov5_saved_model version: 1}
2022-07-30 11:10:08.153468: I tensorflow_serving/core/loader_harness.cc:66] Approving load for servable version {name: yolov5_saved_model version: 1}
2022-07-30 11:10:08.153495: I tensorflow_serving/core/loader_harness.cc:74] Loading servable version {name: yolov5_saved_model version: 1}
2022-07-30 11:10:08.153563: I external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:38] Reading SavedModel from: /models/models/model1/1
2022-07-30 11:10:08.221053: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:277] SavedModel load for tags { serve }; Status: fail: Out of range: Read less bytes than requested. Took 67495 microseconds.
2022-07-30 11:10:08.221079: E tensorflow_serving/util/retrier.cc:37] Loading servable: {name: yolov5_saved_model version: 1} failed: Out of range: Read less bytes than requested
2022-07-30 11:11:08.221228: I tensorflow_serving/util/retrier.cc:33] Retrying of Loading servable: {name: yolov5_saved_model version: 1} retry: 1
2022-07-30 11:11:08.221377: I external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:38] Reading SavedModel from: /models/models/model1/1
2022-07-30 11:11:08.281805: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:277] SavedModel load for tags { serve }; Status: fail: Out of range: Read less bytes than requested. Took 60430 microseconds.
2022-07-30 11:11:08.281849: E tensorflow_serving/util/retrier.cc:37] Loading servable: {name: yolov5_saved_model version: 1} failed: Out of range: Read less bytes than requested
2022-07-30 11:12:08.281994: I tensorflow_serving/util/retrier.cc:33] Retrying of Loading servable: {name: yolov5_saved_model version: 1} retry: 2
2022-07-30 11:12:08.282124: I external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:38] Reading SavedModel from: /models/models/model1/1
2022-07-30 11:12:08.316331: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:277] SavedModel load for tags { serve }; Status: fail: Out of range: Read less bytes than requested. Took 34210 microseconds.
2022-07-30 11:12:08.316359: E tensorflow_serving/util/retrier.cc:37] Loading servable: {name: yolov5_saved_model version: 1} failed: Out of range: Read less bytes than requested
2022-07-30 11:13:08.316498: I tensorflow_serving/util/retrier.cc:33] Retrying of Loading servable: {name: yolov5_saved_model version: 1} retry: 3
2022-07-30 11:13:08.316644: I external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:38] Reading SavedModel from: /models/models/model1/1
2022-07-30 11:13:08.348579: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:277] SavedModel load for tags { serve }; Status: fail: Out of range: Read less bytes than requested. Took 31936 microseconds.
2022-07-30 11:13:08.348606: E tensorflow_serving/util/retrier.cc:37] Loading servable: {name: yolov5_saved_model version: 1} failed: Out of range: Read less bytes than requested
2022-07-30 11:14:08.348747: I tensorflow_serving/util/retrier.cc:33] Retrying of Loading servable: {name: yolov5_saved_model version: 1} retry: 4
2022-07-30 11:14:08.348881: I external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:38] Reading SavedModel from: /models/models/model1/1
2022-07-30 11:14:08.381418: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:277] SavedModel load for tags { serve }; Status: fail: Out of range: Read less bytes than requested. Took 32539 microseconds.
2022-07-30 11:14:08.381447: E tensorflow_serving/util/retrier.cc:37] Loading servable: {name: yolov5_saved_model version: 1} failed: Out of range: Read less bytes than requested

ubuntu% 2022-07-30 11:15:08.381606: I tensorflow_serving/util/retrier.cc:33] Retrying of Loading servable: {name: yolov5_saved_model version: 1} retry: 5
2022-07-30 11:15:08.381749: I external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:38] Reading SavedModel from: /models/models/model1/1
2022-07-30 11:15:08.413642: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:277] SavedModel load for tags { serve }; Status: fail: Out of range: Read less bytes than requested. Took 31895 microseconds.
2022-07-30 11:15:08.413673: E tensorflow_serving/util/retrier.cc:37] Loading servable: {name: yolov5_saved_model version: 1} failed: Out of range: Read less bytes than requested
2022-07-30 11:15:08.413680: I tensorflow_serving/util/retrier.cc:46] Retrying of Loading servable: {name: yolov5_saved_model version: 1} exhausted max_num_retries: 5
2022-07-30 11:15:08.413698: I tensorflow_serving/core/loader_harness.cc:155] Encountered an error for servable version {name: yolov5_saved_model version: 1}: Out of range: Read less bytes than requested
2022-07-30 11:15:08.413705: E tensorflow_serving/core/aspired_versions_manager.cc:388] Servable {name: yolov5_saved_model version: 1} cannot be loaded: Out of range: Read less bytes than requested
Failed to start server. Error: Unknown: 1 servable(s) did not become available: {{{name: yolov5_saved_model version: 1} due to error: Out of range: Read less bytes than requested}, }
2022-07-30 11:15:09.097717: I tensorflow_serving/core/basic_manager.cc:279] Unload all remaining servables in the manager.

[1]  + exit 255   docker run --rm -p 8500:8500 --mount  -t tensorflow/serving:latest-gpu
ubuntu%
```![请添加图片描述](https://img-blog.csdnimg.cn/1f8990d3771e4eeea98e3eb1bddd1f24.png)