Pytorch--3--TX2中源码编译并安装pytorch1.0（亲自操作并成功！）

最新推荐文章于 2024-06-01 08:37:04 发布

明宇朗心

最新推荐文章于 2024-06-01 08:37:04 发布

阅读量6k

点赞数 3

分类专栏： pytorch 文章标签： pytorch1.0 安装编译流程 TX2 pytorch

本文链接：https://blog.csdn.net/qq_33869371/article/details/88591538

版权

pytorch 专栏收录该内容

3 篇文章 1 订阅

订阅专栏

Yolov-1-TX2上用YOLOv3训练自己数据集的流程(VOC2007-TX2-GPU)

Yolov--2--一文全面了解深度学习性能优化加速引擎---TensorRT

Yolov--3--TensorRT中yolov3性能优化加速（基于caffe）

yolov-5-目标检测：YOLOv2算法原理详解

yolov--8--Tensorflow实现YOLO v3

yolov--9--YOLO v3的剪枝优化

yolov--10--目标检测模型的参数评估指标详解、概念解析

yolov--11--YOLO v3的原版训练记录、mAP、AP、recall、precision、time等评价指标计算

yolov--12--YOLOv3的原理深度剖析和关键点讲解

注意！！！一定要注意版本问题，太低的话需要更新，否则会安装失败！（含泪实践!）

本机的版本为：(参照另一篇博客：https://blog.csdn.net/qq_33869371/article/details/88375480）

当前博主的TX2更新的版本为：Jetpack 3.3, cuda 9.0.252, cudnn7.0, opencv3.3.1, TensorRT4.0.2,

系统内核：tegra-ubuntu 4.4.38-tegra aarch64， Linux系统版本：Ubuntu16.04，，cmake 3.5.1

更新后为Python3.5.2（原刷机后为Python：2.7.11+），pip19.0.3，setuptools-40.8.0(原刷机后为setuptools-36.6.0), 这3个的安装见：https://blog.csdn.net/qq_33869371/article/details/88106059 （中间有好几个坑！注意哈！）

旧版本操作没能成功:问题及报错在另一篇博客：https://blog.csdn.net/qq_33869371/article/details/88168202

如何开辟1G的虚拟内存，并开启最大功耗模式？---见前面部分： https://blog.csdn.net/qq_33869371/article/details/87706617

TX2作为一个嵌入式平台的深度学习端，具备不错的GPU性能，我们可以发现TX2的GPU的计算能力是6.2。这意味着TX2对半精度运算有着良好的支持，我们完全可以在桌面端训练好模型，然后移植到TX2上利用半精度运行进行推理，这样可以达到生产落地的效果。

如何在TX2中源码编译Pytorch-1.0？

首先我们需要一个相对纯净的jetpack系统，3.2-3.3版本(最新的4.1.1也可以)都可以，

所以我们最好将TX2的系统重新刷一遍，以免造成一些其他不兼容的错误！！！

刷机教程见另一篇博客：https://blog.csdn.net/qq_33869371/article/details/88429685

《在NVIDIA Jetson TX2上源码编译并使用Pytorch》

步骤

接下来我们严格按照步骤来进行Pytorch的源码安装。

在JetPack-3.2系统中，我们的python一般有两个版本，python命令对应着python2.7。而python3命令对应着python3.5。这里我们使用python3作为编译环境，大家一定要分清楚这两个不同python版本的命令集，否则会造成编译错误。

可以通过which python3来查看当前的python3.5的执行环境。

依赖件

首先安装依赖件：

注意我们使用的命令pip3对应着python3，如果你不清楚系统中的pip和python的绑定信息，采用pip --version查看当前命令在哪个python中，例如我这边则是：

pip3 --version
pip3 9.0.1 from path/to/lib/python3.5/site-packages/pip (python 3.5)

总之我们要使用对应python3的pip命令，首先安装pip3，然后在python3环境中安装一些必要组件。

sudo apt install libopenblas-dev libatlas-dev liblapack-dev
sudo apt install liblapacke-dev checkinstall # For OpenCV
sudo apt-get install python3-pip

pip3 install --upgrade pip3==9.0.1
sudo apt-get install python3-dev

sudo pip3 install numpy scipy # 这个花费的时间略长，20分钟到30分钟左右
sudo pip3 install pyyaml
sudo pip3 install scikit-build
sudo apt-get -y install cmake
sudo apt install libffi-dev
sudo pip3 install cffi

安装完之后，我们添加cudnn的lib和include路径，为什么要执行这一步，因为我们在刷好机后，cuda和cudnn也已经安装好，但是JetPack系统中的cudnn路径和我们一般ubuntu系统中的路径略有不同(为什么不同看这里：https://oldpan.me/archives/pytorch-gpu-ubuntu-nvidia-cuda90)，这时需要我们将cudnn的路径添加到环境变量中并激活：

sudo gedit ~/.bashrc
export CUDNN_LIB_DIR=/usr/lib/aarch64-linux-gnu
export CUDNN_INCLUDE_DIR=/usr/include
source ~/.bashrc

下载Pytorch源码包

我们从github上直接拷贝最新的Pytorch源码包，然后利用pip3安装好所有必备的库，并对第三方库进行更新。

git clone http://github.com/pytorch/pytorch
cd pytorch
sudo pip3 install -U setuptools
sudo pip3 install -r requirements.txt
git submodule update --init --recursive

编译

在编译之前，我们先开启TX2的最大功率模式，这样可以使我们的编译速度稍微快一些：

sudo nvpmodel -m 0         # 切换工作模式到最大
sudo  ~/jetson_clocks.sh   # 强制开启风扇最大转速

这样风扇就开始呼呼转了，我们开始编译吧。

进入Pytorch源码目录后，我们首先执行下面这一句首先编译Pytorch的开发组件：

python3 setup.py build_deps

漫长的编译后(2小时)，我们继续执行以下的命令：

NO_SYSTEM_NCCL=1 DEBUG=1 sudo python3 setup.py build develop

《在NVIDIA Jetson TX2上源码编译并使用Pytorch》

同样是漫长的编译(2小时)，等待后不出意外就编译成功了！

Copying extension caffe2.python.caffe2_pybind11_state_gpu
Copying caffe2.python.caffe2_pybind11_state_gpu from build/caffe2/python/caffe2_pybind11_state_gpu.cpython-35m-aarch64-linux-gnu.so to /home/nvidia/software/pytorch-stable/build/lib.linux-aarch64-3.5/caffe2/python/caffe2_pybind11_state_gpu.cpython-35m-aarch64-linux-gnu.so
copying build/lib.linux-aarch64-3.5/torch/_C.cpython-35m-aarch64-linux-gnu.so -> torch
copying build/lib.linux-aarch64-3.5/torch/_dl.cpython-35m-aarch64-linux-gnu.so -> torch
copying build/lib.linux-aarch64-3.5/torch/_nvrtc.cpython-35m-aarch64-linux-gnu.so -> torch
copying build/lib.linux-aarch64-3.5/caffe2/python/caffe2_pybind11_state.cpython-35m-aarch64-linux-gnu.so -> caffe2/python
copying build/lib.linux-aarch64-3.5/caffe2/python/caffe2_pybind11_state_gpu.cpython-35m-aarch64-linux-gnu.so -> caffe2/python
Creating /usr/local/lib/python3.5/dist-packages/torch.egg-link (link to .)
Adding torch 1.1.0a0 to easy-install.pth file
Installing convert-caffe2-to-onnx script to /usr/local/bin
Installing convert-onnx-to-caffe2 script to /usr/local/bin

Installed /home/nvidia/software/pytorch-stable
Processing dependencies for torch==1.1.0a0
Finished processing dependencies for torch==1.1.0a0
WARNING: 'develop' is not building C++ code incrementally
because ninja is not installed. Run this to enable it:
 > pip install ninja

后续操作

后续我们继续安装Pytorch推荐搭配的组件torchvision。

sudo apt clean
sudo apt-get install libjpeg-dev zlib1g-dev

git clone https://github.com/python-pillow/Pillow.git
cd Pillow/
sudo python3 setup.py install
sudo pip3 install pandas # ~20-30 min
sudo pip3 install Cython
sudo pip3 install scikit-image

sudo pip3 --no-cache-dir install torchvision

遇到的问题：

执行：sudo python3 setup.py install 出现以下问题：

nvidia@tegra-ubuntu:~/software/Pillow$ sudo python3 setup.py install
running install
running bdist_egg
running egg_info
writing dependency_links to src/Pillow.egg-info/dependency_links.txt
writing top-level names to src/Pillow.egg-info/top_level.txt
writing src/Pillow.egg-info/PKG-INFO
reading manifest file 'src/Pillow.egg-info/SOURCES.txt'
reading manifest template 'MANIFEST.in'
warning: no files found matching '*.c'
warning: no files found matching '*.h'
warning: no files found matching '*.sh'
warning: no previously-included files found matching '.appveyor.yml'
warning: no previously-included files found matching '.coveragerc'
warning: no previously-included files found matching '.codecov.yml'
warning: no previously-included files found matching '.editorconfig'
warning: no previously-included files found matching '.landscape.yaml'
warning: no previously-included files found matching '.readthedocs.yml'
warning: no previously-included files found matching 'azure-pipelines.yml'
warning: no previously-included files found matching 'tox.ini'
warning: no previously-included files matching '*.pyc' found anywhere in distribution
warning: no previously-included files matching '*.so' found anywhere in distribution
no previously-included directories found matching '.azure-pipelines'
no previously-included directories found matching '.travis'
writing manifest file 'src/Pillow.egg-info/SOURCES.txt'
installing library code to build/bdist.linux-aarch64/egg
running install_lib
running build_py
running build_ext


The headers or library files could not be found for jpeg,
a required dependency when compiling Pillow from source.

Please see the install instructions at:
   https://pillow.readthedocs.io/en/latest/installation.html

Traceback (most recent call last):
  File "setup.py", line 793, in <module>
    zip_safe=not (debug_build() or PLATFORM_MINGW), )
  File "/usr/local/lib/python3.5/dist-packages/setuptools/__init__.py", line 145, in setup
    return distutils.core.setup(**attrs)
  File "/usr/lib/python3.5/distutils/core.py", line 148, in setup
    dist.run_commands()
  File "/usr/lib/python3.5/distutils/dist.py", line 955, in run_commands
    self.run_command(cmd)
  File "/usr/lib/python3.5/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/usr/local/lib/python3.5/dist-packages/setuptools/command/install.py", line 67, in run
    self.do_egg_install()
  File "/usr/local/lib/python3.5/dist-packages/setuptools/command/install.py", line 109, in do_egg_install
    self.run_command('bdist_egg')
  File "/usr/lib/python3.5/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python3.5/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/usr/local/lib/python3.5/dist-packages/setuptools/command/bdist_egg.py", line 172, in run
    cmd = self.call_command('install_lib', warn_dir=0)
  File "/usr/local/lib/python3.5/dist-packages/setuptools/command/bdist_egg.py", line 158, in call_command
    self.run_command(cmdname)
  File "/usr/lib/python3.5/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python3.5/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/usr/local/lib/python3.5/dist-packages/setuptools/command/install_lib.py", line 11, in run
    self.build()
  File "/usr/lib/python3.5/distutils/command/install_lib.py", line 109, in build
    self.run_command('build_ext')
  File "/usr/lib/python3.5/distutils/cmd.py", line 313, in run_command
    self.distribution.run_command(command)
  File "/usr/lib/python3.5/distutils/dist.py", line 974, in run_command
    cmd_obj.run()
  File "/usr/lib/python3.5/distutils/command/build_ext.py", line 338, in run
    self.build_extensions()
  File "setup.py", line 606, in build_extensions
    raise RequiredDependencyException(f)
__main__.RequiredDependencyException: jpeg

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "setup.py", line 805, in <module>
    raise RequiredDependencyException(msg)
__main__.RequiredDependencyException: 

The headers or library files could not be found for jpeg,
a required dependency when compiling Pillow from source.

Please see the install instructions at:
   https://pillow.readthedocs.io/en/latest/installation.html

测试Pytorch是否安装成功

编译完成之后，如果安装成功，会出现下面的信息：

《在NVIDIA Jetson TX2上源码编译并使用Pytorch》

需要注意的点

在JetPack系统中，我们不论是执行编译还是执行代码，都必须加上sudo，否则会出现编译失败或者cuda error的情况。

参考

https://m.oldpan.me/archives/nvidia-jetson-tx2-source-build-pytorch

https://github.com/Bfzanchetta/DLARM/blob/master/jetsontx2/pytorch_install.sh

明宇朗心

关注

3
点赞
踩
23

收藏

觉得还不错? 一键收藏
打赏
28
评论
Pytorch--3--TX2中源码编译并安装pytorch1.0（亲自操作并成功！）

Yolov-1-TX2上用YOLOv3训练自己数据集的流程(VOC2007-TX2-GPU)Yolov--2--一文全面了解深度学习性能优化加速引擎---TensorRTYolov--3--TensorRT中yolov3性能优化加速（基于caffe）yolov-5-目标检测：YOLOv2算法原理详解yolov--8--Tensorflow实现YOLO v3yolov--9--Y...
复制链接

扫一扫