小小甜菜tensorrt爬坑记
遍观全球,模型优化部署方案中生产化最好的是intel,英伟达,谷歌(TPU)。我在研究了intel的openvino后,有感于CPU和VPU的缺陷,准备使用tensorrt作为补充。
安装
sudo apt-get install nvidia-driver-418
sudo chmod +x cuda_10.0.130_410.48_linux.run
sudo ./cuda_10.0.130_410.48_linux.run
sudo gedit ~/.bashrc
export CUDA_HOME=/usr/local/cuda
export PATH=$PATH:$CUDA_HOME/bin
export LD_LIBRARY_PATH=/usr/local/cuda-10.0/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
source ~/.bashrc
tar -xzvf cudnn-10.0-linux-x64-v7.6.3.30.tgz
sudo cp cuda/include/cudnn.h /usr/local/cuda/include/
sudo cp cuda/lib64/libcudnn* /usr/local/cuda/lib64/
sudo chmod a+r /usr/local/cuda/include/cudnn.h
sudo chmod a+r /usr/local/cuda/lib64/libcudnn*
sudo apt install python3-pip
pip3 install numpy -i https://pypi.tuna.tsinghua.edu.cn/simple
sudo apt-get install python3.6-dev
pip3 install pycuda -i https://pypi.tuna.tsinghua.edu.cn/simple
//官网下载对于版本
tar xzvf TensorRT-7.0.0.11.Ubuntu-18.04.x86_64-gnu.cuda-10.0.cudnn7.6.tar.gz
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/xs/TensorRT-7.0.0.11/lib
cd TensorRT-7.0.0.11/python
sudo pip3 install tensorrt-7.0.0.11-cp36-none-linux_x86_64.whl
cd ../uff
sudo pip3 install uff-0.6.5-py2.py3-none-any.whl -i https://pypi.tuna.tsinghua.edu.cn/simple
cd ../graphsurgeon
sudo pip3 install graphsurgeon-0.4.1-py2.py3-none-any.whl
pip3 install tensorflow-gpu -i https://pypi.tuna.tsinghua.edu.cn/simple
cd ../lib
sudo cp libnvinfer.so.7 /usr/lib/
sudo cp libnvonnxparser.so.7 /usr/lib/
sudo cp libmyelin.so.1 /usr/lib/
sudo cp libnvparsers.so.7 /usr/lib/
sudo cp libnvinfer_plugin.so.7 /usr/lib/
ONNX框架
使用tensorrt就不得不提到微软和facebook合力创建的开源社区平台ONNX。不过目前tensorrt5.1.5版本不支持opset10,我会继续跟踪。
sudo apt-get install python3-pip protobuf-compiler libprotoc-dev
pip3 install onnx -i https://pypi.tuna.tsinghua.edu.cn/simple
pip3 install onnxruntime-gpu -i https://pypi.tuna.tsinghua.edu.cn/simple
onnx-tensorrt
首先升级cmake到3.13
sudo apt-get install libprotobuf-dev protobuf-compiler
git clone --recurse-submodules https://github.com/onnx/onnx-tensorrt.git
mkdir build
cd build
cmake .. -DTENSORRT_ROOT=/home/xs/TensorRT-7.0.0.11
make -j8
sudo make install
tensrrt5
由于过去项目不兼容,我重新使用了tensrrt5
//官网下载对于版本
tar xzvf TensorRT-5.1.5.0.Ubuntu-18.04.2.x86_64-gnu.cuda-10.0.cudnn7.5.tar.gz
export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:/home/xs/TensorRT-5.1.5.0/lib
cd TensorRT-5.1.5.0/python
sudo pip3 install tensorrt-5.1.5.0-cp36-none-linux_x86_64.whl
cd ../uff
sudo pip3 install uff-0.6.5-py2.py3-none-any.whl -i https://pypi.tuna.tsinghua.edu.cn/simple
cd ../graphsurgeon
sudo pip3 install graphsurgeon-0.4.1-py2.py3-none-any.whl
pip3 install tensorflow-gpu -i https://pypi.tuna.tsinghua.edu.cn/simple
cd ../lib
sudo cp libnvinfer.so.5 /usr/lib/
sudo cp libnvonnxparser.so.0 /usr/lib/
sudo cp libnvonnxparser_runtime.so.0 /usr/lib/
sudo cp libnvparsers.so.5 /usr/lib/
sudo cp libnvinfer_plugin.so.5 /usr/lib/
#onnx-tensorrt
git clone -b v5.0 --recurse-submodules https://github.com/onnx/onnx-tensorrt.git
tensorrt-inference-server
这是tensorrt的服务器部署方案
docker pull nvcr.io/nvidia/tensorrtserver:19.08-py3
deepstream
这是tensorrt的嵌入式部署方案