生成tensorrt引擎错误记录-yolov5

置顶 Coding-Prince

已于 2022-07-06 11:26:35 修改

阅读量1w

点赞数 13

文章标签： c++ 深度学习

于 2021-05-06 19:58:03 首次发布

本文链接：https://blog.csdn.net/qq_42362891/article/details/116107108

版权

warning: ‘nvinfer1::Dims::type’ is deprecated [-Wdeprecated-declarations]
note:TRT_DEPRECATED DimensionType type[MAX_DIMS]; //!< The type of each dimensi
执行make时出错：/usr/bin/ld: cannot find -lnvinfer
解决方式：cmakelists添加
include_directories("/home/tensorrt_tar/TensorRT-6.0.1.5/include") #具体路径根据实际安装路径来写
link_directories(/home/tensorrt_tar/TensorRT-6.0.1.5/lib)

背景介绍：生成engine ，模型yolov5，ubantu16.04

具体流程：

2. build tensorrtx/yolov5 and run
// put yolov5s.wts into tensorrtx/yolov5
// go to tensorrtx/yolov5
// update CLASS_NUM in yololayer.h if your model is trained on custom dataset
mkdir build
cd build
cmake ..
make
sudo ./yolov5 -s [.wts] [.engine] [s/m/l/x or c gd gw]  // serialize model to plan file
sudo ./yolov5 -d [.engine] [image folder]  // deserialize and run inference, the images in [image folder] will be processed.
// For example yolov5s
sudo ./yolov5 -s yolov5s.wts yolov5s.engine s
sudo ./yolov5 -d yolov5s.engine ../samples
// For example Custom model with depth_multiple=0.17, width_multiple=0.25 in yolov5.yaml
sudo ./yolov5 -s yolov5_custom.wts yolov5.engine c 0.17 0.25
sudo ./yolov5 -d yolov5.engine ../samples

cmake..时：

1.CMake Warning at CMakeLists.txt:27 (find_package):
By not providing "FindOpenCV.cmake" in CMAKE_MODULE_PATH this project has
asked CMake to find a package configuration file provided by "OpenCV", but
CMake did not find one.

cmakelists中添加：set(OpenCV_DIR /home/.conda/envs/pytorch/share/OpenCV/)

make时：

1.fatal error: NvInfer.h:没有那个文件或者目录.compilation terminated.

解决方式：cmakelists添加

include_directories("/home/tensorrt_tar/TensorRT-6.0.1.5/include") #具体路径根据实际安装路径来写
link_directories(/home/tensorrt_tar/TensorRT-6.0.1.5/lib)

2./usr/bin/ld: 找不到-lnvinfer
问题分析：其实还是库路径问题，对于新手而言，由于不明白cmakelists里的代码含义，在cmakelists里添加库路径时，不知道应该添加在什么位置。比如我，就照猫画虎的添加在了

find_package(OpenCV)

这句代码之前，结果make时就一直报：找不到-lnvinfer。明明之前出现这样的问题，添加路径就可以解决呀，现在怎么不行呢？
后来我发现，添加路径的代码要写在合适的位置，简单来说就是必须先设置库路径，再调用库路径里的东西（这是句废话，但新手有时候就是容易忽略掉这样简单的逻辑）。此外，还有一个关键问题是什么是调用库相关的代码？目前我见到的有

cuda_add_library(myplugins SHARED ${PROJECT_SOURCE_DIR}/yololayer.cu)
target_link_libraries(myplugins nvinfer cudart)

之前报错的原因就是因为我在它们之后才添加的路径。

sudo ./yolov5 -s [.wts] [.engine]时：

1./common.hpp:156: nvinfer1::IScaleLayer* addBatchNorm2d(nvinfer1::INetworkDefinition*, std::map<std::__cxx11::basic_string<char>, nvinfer1::Weights>&, nvinfer1::ITensor&, std::__cxx11::string, float): Assertion `scale_1' failed.
问题分析：其实这个错误的关键还是在最后一句 Assertion `scale_1' failed.
定位到代码中


IScaleLayer* scale_1 = network->addScale(input, ScaleMode::kCHANNEL, shift, scale, power);
assert(scale_1);

Loading weights: yolov5s.wts
[05/07/2021-10:05:06] [E] [TRT] Parameter check failed at: ../builder/Network.cpp::addScale::286, condition: shift.count > 0 ? (shift.values != nullptr) : (shift.values == nullptr)

问题分析：提示断言scale_1有问题，说明scale_1这个层生成的不对，说明network->addScale的input, ScaleMode::kCHANNEL, shift, scale, power这些参数可能存在问题，具体哪个需要进一步调试，可能性相对较大的原因是没有正确取到网络的权重。
我的问题原因是没有按照readme规范操作，没有取到正确的权重。
问题解析：readme的流程少看了一段，如下

1. generate .wts from pytorch with .pt, or download .wts from model zoo

```
// git clone src code according to `Different versions of yolov5` above
// download https://github.com/ultralytics/yolov5/releases/download/v4.0/yolov5s.pt
// copy tensorrtx/yolov5/gen_wts.py into ultralytics/yolov5
// ensure the file name is yolov5s.pt and yolov5s.wts in gen_wts.py
// go to ultralytics/yolov5
python gen_wts.py
// a file 'yolov5s.wts' will be generated.
```

其中，生成wts文件时，我直接在pycharm运行gen_wts.py这个文件，结果报错
result = unpickler.load()
ModuleNotFoundError: No module named 'models'
我以为是环境错误，然后做了个骚操作，利用yolov5 detect.py构建的网络来生成wts文件，ok，成功生成，但是接下来生成引擎时就是无尽的 shift.count > 0 ? (shift.values != nullptr) : (shift.values == nullptr)了。

解决方式：按照官方操作来！！！

由gen_wts.py文件生成wts文件
gen_wts.py里需要指定yolov5权重.pt文件的路径，可以是绝对路径，可以是相对路径
如果是相对路径，需要把gen_wts.py拷贝到yolov5权重所在文件
注意tensorrtx-yolov5和yolov5权重版本的对应
官方操作就是指README.md文件里给出的操作

我的代码示例如下：

import torch
import struct
from utils.torch_utils import select_device

# Initialize
device = select_device('cpu')
/*
只需要修改pt和wts路径即可
*/
# Load model yolov5权重所在路径，这里是相对路径
model = torch.load('weights/yolov5s.pt', map_location=device)['model'].float()  # load to FP32
model.to(device).eval()

with open('yolov5s.wts', 'w') as f:#生成的wts路径
    f.write('{}\n'.format(len(model.state_dict().keys())))
    for k, v in model.state_dict().items():
        vr = v.reshape(-1).cpu().numpy()
        f.write('{} {} '.format(k, len(vr)))
        for vv in vr:
            f.write(' ')
            f.write(struct.pack('>f',float(vv)).hex())
        f.write('\n')

tensorrtx-yolov5代码地址：https://github.com/ultralytics/yolov5.git，里面有不同yolov5版本的tensorrtx代码，我采用的是tensorrtx-yolov5-v4。

Currently, we support yolov5 v1.0(yolov5s only), v2.0, v3.0, v3.1 and v4.0.

- For yolov5 v4.0, download .pt from [yolov5 release v4.0](https://github.com/ultralytics/yolov5/releases/tag/v4.0), `git clone -b v4.0 https://github.com/ultralytics/yolov5.git` and `git clone https://github.com/wang-xinyu/tensorrtx.git`, then follow how-to-run in current page.
- For yolov5 v3.1, download .pt from [yolov5 release v3.1](https://github.com/ultralytics/yolov5/releases/tag/v3.1), `git clone -b v3.1 https://github.com/ultralytics/yolov5.git` and `git clone -b yolov5-v3.1 https://github.com/wang-xinyu/tensorrtx.git`, then follow how-to-run in [tensorrtx/yolov5-v3.1](https://github.com/wang-xinyu/tensorrtx/tree/yolov5-v3.1/yolov5).
- For yolov5 v3.0, download .pt from [yolov5 release v3.0](https://github.com/ultralytics/yolov5/releases/tag/v3.0), `git clone -b v3.0 https://github.com/ultralytics/yolov5.git` and `git clone -b yolov5-v3.0 https://github.com/wang-xinyu/tensorrtx.git`, then follow how-to-run in [tensorrtx/yolov5-v3.0](https://github.com/wang-xinyu/tensorrtx/tree/yolov5-v3.0/yolov5).
- For yolov5 v2.0, download .pt from [yolov5 release v2.0](https://github.com/ultralytics/yolov5/releases/tag/v2.0), `git clone -b v2.0 https://github.com/ultralytics/yolov5.git` and `git clone -b yolov5-v2.0 https://github.com/wang-xinyu/tensorrtx.git`, then follow how-to-run in [tensorrtx/yolov5-v2.0](https://github.com/wang-xinyu/tensorrtx/tree/yolov5-v2.0/yolov5).
- For yolov5 v1.0, download .pt from [yolov5 release v1.0](https://github.com/ultralytics/yolov5/releases/tag/v1.0), `git clone -b v1.0 https://github.com/ultralytics/yolov5.git` and `git clone -b yolov5-v1.0 https://github.com/wang-xinyu/tensorrtx.git`, then follow how-to-run in [tensorrtx/yolov5-v1.0](https://github.com/wang-xinyu/tensorrtx/tree/yolov5-v1.0/yolov5).