【TensorRT】tensorRT 7.0的安装配置

前几天老黄发布了tensorRT 7.0, 对于模型服务器部署来说，tensorRT是真的香啊。先放官网链接，https://docs.nvidia.com/deeplearning/sdk/tensorrt-developer-guide/index.html。

先介绍一下TensorRT

TensorRT的核心就是一个深度学习的高性能C++的推理库， 基本适配主流的训练框架，如Tensorflow, Caffe, Pytorch, MXNet。

Some training frameworks such as TensorFlow have integrated TensorRT so that it can be used to accelerate inference within the framework. Alternatively, TensorRT can be used as a library within a user application. It includes parsers for importing existing models from Caffe, ONNX, or TensorFlow, and C++ and Python APIs for building models programmatically.

TensorRT 7.0的安装

这里介绍在ubuntu下的两个安装方式：deb和tar安装。

一、deb安装：

注意deb安装需要sudo权限，没有sudo权限的下面可以不用看，直接看第二种安装方式。

1. 先从 https://developer.nvidia.com/tensorrt 官网下载一个 tensorRT 的 deb版本， 这里跟下载cudnn一样， 需要进行验证才能下载

注意本博客介绍的第一种安装方式需要下载deb的包，tar是第二种安装方式，建议先把两个都下载下来。

2）按照TensorRT的版本：这里nvidia的官网写的很繁琐，可以直接dpkg -i 安装下载的deb安装包，然后记得把key加进来

os=”ubuntu1x04”
tag=”cudax.x-trt7.x.x.x-ga-yyyymmdd”
sudo dpkg -i nv-tensorrt-repo-${os}-${tag}_1-1_amd64.deb
sudo apt-key add /var/nv-tensorrt-repo-${tag}/7fa2af80.pub

sudo apt-get update
sudo apt-get install tensorrt

If using Python 2.7:

sudo apt-get install python-libnvinfer-dev

The following additional packages will be installed:

python-libnvinfer

If using Python 3.x:

sudo apt-get install python3-libnvinfer-dev

The following additional packages will be installed:

python3-libnvinfer

If you plan to use TensorRT with TensorFlow:

sudo apt-get install uff-converter-tf

The graphsurgeon-tf package will also be installed with the above command.

3。 验证一下是否安装完毕：

dpkg -l | grep TensorRT

You should see something similar to the following:

ii  graphsurgeon-tf	7.0.0-1+cuda10.2	amd64	GraphSurgeon for TensorRT package
ii  libnvinfer-bin		7.0.0-1+cuda10.2	amd64	TensorRT binaries
ii  libnvinfer-dev		7.0.0-1+cuda10.2	amd64	TensorRT development libraries and headers
ii  libnvinfer-doc		7.0.0-1+cuda10.2	all	TensorRT documentation
ii  libnvinfer-plugin-dev	7.0.0-1+cuda10.2	amd64	TensorRT plugin libraries
ii  libnvinfer-plugin7	7.0.0-1+cuda10.2	amd64	TensorRT plugin libraries
ii  libnvinfer-samples	7.0.0-1+cuda10.2	all	TensorRT samples
ii  libnvinfer7		7.0.0-1+cuda10.2	amd64	TensorRT runtime libraries
ii  libnvonnxparsers-dev		7.0.0-1+cuda10.2	amd64	TensorRT ONNX libraries
ii  libnvonnxparsers7	7.0.0-1+cuda10.2	amd64	TensorRT ONNX libraries
ii  libnvparsers-dev	7.0.0-1+cuda10.2	amd64	TensorRT parsers libraries
ii  libnvparsers7	7.0.0-1+cuda10.2	amd64	TensorRT parsers libraries
ii  python-libnvinfer	7.0.0-1+cuda10.2	amd64	Python bindings for TensorRT
ii  python-libnvinfer-dev	7.0.0-1+cuda10.2	amd64	Python development package for TensorRT
ii  python3-libnvinfer	7.0.0-1+cuda10.2	amd64	Python 3 bindings for TensorRT
ii  python3-libnvinfer-dev	7.0.0-1+cuda10.2	amd64	Python 3 development package for TensorRT
ii  tensorrt		7.0.0.x-1+cuda10.2 	amd64	Meta package of TensorRT
ii  uff-converter-tf	7.0.0-1+cuda10.2	amd64	UFF converter for TensorRT package

这样就安装完毕。

注意： 我在安装过程中，则出现了问题：

Reading package lists... Done
Building dependency tree       
Reading state information... Done
Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
 tensorrt : Depends: libnvinfer7 (= 7.0.0-1+cuda10.0) but it is not going to be installed
            Depends: libnvinfer-plugin7 (= 7.0.0-1+cuda10.0) but it is not going to be installed
            Depends: libnvparsers7 (= 7.0.0-1+cuda10.0) but it is not going to be installed
            Depends: libnvonnxparsers7 (= 7.0.0-1+cuda10.0) but it is not going to be installed
            Depends: libnvinfer-bin (= 7.0.0-1+cuda10.0) but it is not going to be installed
            Depends: libnvinfer-dev (= 7.0.0-1+cuda10.0) but it is not going to be installed
            Depends: libnvinfer-plugin-dev (= 7.0.0-1+cuda10.0) but it is not going to be installed
            Depends: libnvparsers-dev (= 7.0.0-1+cuda10.0) but it is not going to be installed
            Depends: libnvonnxparsers-dev (= 7.0.0-1+cuda10.0) but it is not going to be installed
            Depends: libnvinfer-samples (= 7.0.0-1+cuda10.0) but it is not going to be installed
            Depends: libnvinfer-doc (= 7.0.0-1+cuda10.0) but it is not going to be installed
E: Unable to correct problems, you have held broken packages.

这好像安装cuda的时候出现了问题，有时间的同学请重新安装cuda，跟我一样比较懒的同学移步看下一个安装方式。

二、tar安装：

这个按照方式比较简单，不需要管理员权限。
对下载好的tar文件进行解压：

version=”7.x.x.x”
os=”<os>”
arch=$(uname -m)
cuda=”cuda-x.x”
cudnn=”cudnn7.x”
tar xzvf TensorRT-${version}.${os}.${arch}-gnu.${cuda}.${cudnn}.tar.gz

然后把lib路径加到系统动态库路径上。

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:<TensorRT-${version}/lib>

解压后：目录如下

ls TensorRT-${version}
bin  data  doc  graphsurgeon  include  lib  python  samples  targets  TensorRT-Release-Notes.pdf  uff

Install the Python TensorRT wheel file.

cd TensorRT-${version}/python

If using Python 2.7:

sudo pip2 install tensorrt-*-cp27-none-linux_x86_64.whl

If using Python 3.x:

sudo pip3 install tensorrt-*-cp3x-none-linux_x86_64.whl

Install the Python UFF wheel file. This is only required if you plan to use TensorRT with TensorFlow.

cd TensorRT-${version}/uff

If using Python 2.7:

sudo pip2 install uff-0.6.5-py2.py3-none-any.whl

If using Python 3.x:

sudo pip3 install uff-0.6.5-py2.py3-none-any.whl

In either case, check the installation with:

which convert-to-uff

Install the Python graphsurgeon wheel file.

cd TensorRT-${version}/graphsurgeon

If using Python 2.7:

sudo pip2 install graphsurgeon-0.4.1-py2.py3-none-any.whl

If using Python 3.x:

sudo pip3 install graphsurgeon-0.4.1-py2.py3-none-any.whl