tensorrt的部署和问题

问题1
模型输入输出不支持字典，将pytorch首先转化为onnx，然后让TensorRT解析onnx从而构建TensorRT引擎。
让TensorRT直接加载引擎文件，也就是说，我们先把onnx转化为TensorRT的trt文件，然后让c++环境下的TensorRT直接加载trt文件，从而构建engine，所以需要将.pth模型文件转换为onnx,当onnx转换为trt时，报错:
ERROR: Failed to parse the ONNX file.
parser.get_error(error) In node 29 (convert_axis): UNSUPPORTED_NODE: Assertion failed: axis >= 0 && axis < nbDims
此处node 29定位问题的方法，需要将onnx模型可视化，直接调用netron库，即可显示

显示内容如下：

第29节点转换报错，第一节点为25，遂在第一节点基础上加29，查找54网络结构如下：
发现出现了并不支持的‘ Gather ’
问题2
Loading ONNX file from path ep_200_res50_batchsize_1_input_output.onnx…
Beginning ONNX file parsing
Completed parsing of ONNX file
layers: 0
python3.6: …/builder/Network.cpp:863: virtual nvinfer1::ILayer* nvinfer1::Network::getLayer(int) const: Assertion `layerIndex >= 0’ failed.
已放弃 (核心已转储)
解决方法：使用管理员权限即可

问题3
TypeError: a bytes-like object is required, not ‘Tensor’**
Traceback (most recent call last):
File “compare_example.py”, line 132, in
img_leaf_evaluate(img)
File “compare_example.py”, line 115, in img_leaf_evaluate
inputs, outputs, bindings, stream = allocate_buffers(engine) # input, output: host # bindings
File “compare_example.py”, line 52, in allocate_buffers
stream = cuda.Stream()
pycuda._driver.LogicError: explicit_context_dependent failed: invalid device context - no currently active context?
原因：pycuda.driver没有初始化，导致无法得到context
解决方法: 需要在导入pycuda.driver后再导入pycuda.autoinit。

问题4
onnx2trt: error while loading shared libraries: libcublas.so.10.0: cannot open shared object file: No such file or directory**

解决方法：…export LD_LIBRARY_PATH= /usr/local/cuda/lib64$ {LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}

问题5
Parsing model WARNING: Your ONNX model has been generated with INT64 weights, while TensorRT does not natively support INT64. Attempting to cast down to INT32. Successfully casted down to INT32. While parsing node number 146 [Gather -> “409”]: ERROR: /h**
解决方法：使用python第三方库简化onnx
python -m onnxsim retinate_hat_hair_beard.onnx retinate_hat_hair_beard_sim.onnx

问题6
[TensorRT] ERROR: Network must have at least one output
解决方法：
在with循环之外增加了两句代码
last_layer = network.get_layer(network.num_layers - 1)
network.mark_output(last_layer.get_output(0))

问题7[TensorRT] ERROR: INVALID_CONFIG: The engine plan file is generated on an incompatible device, expecting compute 5.3 got compute 7.5, please rebuild.
解决方法：用7.5计算力生成的trt文件，是不能用6.5的显卡来解析的，需要在目标显卡上生成trt

tips：
pytorch模型转化成的onnx本身就有问题。经常会出现，expand, Gather, reshape不支持等