1.寻找最新开源的ONNX模型
https://github.com/PINTO0309/PINTO_model_zoo/tree/main
这个大佬把2023年新开源的模型都转成ONNX了,很好用。
2.安装ONNX2TensorRT
git clone --recurse-submodules https://github.com/onnx/onnx-tensorrt.git
# Build from source
cd onnx-tensorrt
mkdir build && cd build
cmake ..
make -j$(nproc)
sudo make install
(1).注意适配版本号
注意这里有个坑,git clone默认下载的是最新版的,onnx-tensorrt,有可能和你的tensorrt版本不合。
解决方法是,我的tensorrt版本为8.4,然后就在这个网址里找打onnx2trt的8.4 版本下载即可
https://github.com/onnx/onnx-tensorrt/releases
dpkg -l | grep TensorRT
这个是查看TensorRT版本号的代码
一般出现这样的错误就是tensorrt版本不合
/home/zhangqi/Documents/Library/onnx-tensorrt/TensorOrWeights.hpp:115:38: error: ‘kUINT8’ is not a member of ‘nvinfer1::DataType’; did you mean ‘kINT8’?
115 | case nvinfer1::DataType::kUINT8: return "UINT8";
| ^~~~~~
| kINT8
/home/zhangqi/Documents/Library/onnx-tensorrt/TensorOrWeights.hpp:118:38: error: ‘kFP8’ is not a member of ‘nvinfer1::DataType’
118 | case nvinfer1::DataType::kFP8: return "FP8";
| ^~~~
In file included from /home/zhangqi/Documents/Library/onnx-tensorrt/trt_utils.hpp:8,
from /home/zhangqi/Documents/Library/onnx-tensorrt/onnx2trt_utils.hpp:10,
from /home/zhangqi/Documents/Library/onnx-tensorrt/onnx2trt_utils.cpp:5:
/home/zhangqi/Documents/Library/onnx-tensorrt/TensorOrWeights.hpp: In member function ‘std::string onnx2trt::TensorOrWeights::getType() const’:
/home/zhangqi/Documents/Library/onnx-tensorrt/TensorOrWeights.hpp:115:38: error: ‘kUINT8’ is not a member of ‘nvinfer1::DataType’; did you mean ‘kINT8’?
115 | case nvinfer1::DataType::kUINT8: return "UINT8";
(2).注意适配合适的cuda
fatal error: crt/host_defines.h: No such file or directory
147 | #include "crt/host_defines.h"
| ^~~~~~~~~~~~~~~~~~~~
出现这种状况,就直接搜索全计算机有没有这个文件,我的这个文件在cuda11.1文件夹里而不是默认的cuda文件夹。于是我把onnx2trt里面CmakeList里的
if (NOT CUDA_TOOLKIT_ROOT_DIR)
set(CUDA_TOOLKIT_ROOT_DIR /usr/local/cuda)
改为
if (NOT CUDA_TOOLKIT_ROOT_DIR)
set(CUDA_TOOLKIT_ROOT_DIR /usr/local/cuda-11.1)
3.转换开始!!!
cd build
/onnx2trt ../decoder.onnx -o decoder.trt