TF2.x+TensorRT 环境配置

TF2.x+TensorRT 环境配置

1.相关依赖下载

请务必注意TensorRT所需要的CUDA，cudnn的版本，以及该版本CUDA所支持的Tensorflow版本：

1. TensorRT : Download
2. CUDA: Download
3. cuDNN: Download

本文中对应版本：
TensorRT(7.2.3.4),
CUDA(11.0),
cuDNN(8.1.0),
TF(2.4),
Anaconda(Py>=3.7);

2.安装步骤

1) CUDA

sudo sh cuda_11.0.2_450.51.05_linux.run

其中，NVIDIA驱动可以通过软件中心更新；

添加到系统路径：

#cuda
export PATH="/usr/local/cuda-11.0/bin:$PATH"
export LD_LIBRARY_PATH="/usr/local/cuda-11.0/lib64:$LD_LIBRARY_PATH"

2) cuDNN

参考链接：官方Install教程

tar -xzvf cudnn-11.2-linux-x64-v8.1.0.77.tgz
sudo cp cuda/include/cudnn*.h /usr/local/cuda/include 
$ sudo cp -P cuda/lib64/libcudnn* /usr/local/cuda/lib64 
$ sudo chmod a+r /usr/local/cuda/include/cudnn*.h /usr/local/cuda/lib64/libcudnn*

3) TensorRT

参考链接：官方Install教程 （TAR安装方式最为方便）

tar xzvf TensorRT-7.2.3.4.Ubuntu-18.04.x86_64-gnu.cuda-11.0.cudnn8.1.tar.gz

#tensorrt
export PATH="/usr/local/TensorRT-7.2.3.4/bin:$PATH"
export LD_LIBRARY_PATH="/usr/local/TensorRT-7.2.3.4/lib:$LD_LIBRARY_PATH"
export LD_LIBRARY_PATH="$LD_LIBRARY_PATH:</usr/local/TensorRT-7.2.3.4/lib>"

#Install the python TensorRT wheel file
cd TensorRT-7.2.3.4/python
pip install tensorrt-*-cp3x-none-linux_x86_64.whl


#Install the Python UFF wheel file
cd TensorRT-7.2.3.4/uff
pip install uff-0.6.9-py2.py3-none-any.whl

#Check the installation with:
which convert-to-uff
export PATH="/home/aoi/.local/bin:$PATH"

#Install the Python graphsurgeon wheel file.
cd TensorRT-7.2.3.4/graphsurgeon
pip install graphsurgeon-0.4.5-py2.py3-none-any.whl

#Install the Python onnx-graphsurgeon wheel file.
cd TensorRT-7.2.3.4/onnx_graphsurgeon
pip install onnx_graphsurgeon-0.3.12-py2.py3-none-any.whl

4) Tensorflow

考虑到CUDA版本，本文选择TF2.4版本

#Install bazelisk
export PATH="/usr/local/bin:$PATH"

git clone https://github.com/tensorflow/tensorflow.git
git checkout r2.4
./configure

这里注意./configure的回答：

CUDA: 11.0
cuDNN: 8.1.0
TensorRT: 7
List of base paths:
/usr/local/cuda,/usr/local/cuda/bin,/usr/local/cuda/lib64,/usr/local/cuda/include,/usr/local/TensorRT-7.2.3.4/bin,/usr/local/TensorRT-7.2.3.4/include,/usr/local/TensorRT-7.2.3.4/lib,/usr/local/TensorRT-7.2.3.4

请严格按照以上版本对应的配置输入，否则可能出现的错误，如：“Could not find any NvInferVersion.h matching version ‘7.2.3.4’ in any subdirectory”等问题；

5) Bazel TF with gpu and tensorRT

sudo bazel build --config=opt --config=cuda //tensorflow:libtensorflow_cc.so