Faster RCNN-PyTorch版本运行记录

Faster RCNN-PyTorch版本运行记录

笔者最近在学习《深度学习之PyTorch物体检测实战》，在运行第四章讲解的Faster RCNN时遇到了 一些问题，特此记录。
第四章使用的源码参考了jwyang/faster-rcnn.pytorch：https://github.com/jwyang/faster-rcnn.pytorch的PyTorch复现工程，依赖项是

• Python 2.7或者3.6
• Pytorch 0.4.0
• CUDA 8.0或者更高

准备工作：
1)首先从GitHub上下载本章所用的代码,地址如下:

git clone git@github.com:dongdonghy/Detection-PyTorch-Notebook.git

(2)然后创建data文件夹。

cd Detection-PyTorch-Notebook/chapter4/faster-rcnn-pytorch && mkdir data

(3)下载PASCAL VOC 2012数据集,在data文件夹下创建软连接。

cd data && ln -s "your data directory" VOC2012

PASCAL VOC 数据集简介:
https://blog.csdn.net/mzpmzk/article/details/88065416
https://blog.csdn.net/zhangjunbob/article/details/52769381
(4)从下面网址下载预训练模型并放到data/pretrained_model/文件夹下。
https://www.dropbox.com/s/s3brpk0bdq60nyb/vgg16_caffe.pth?dl=0
(5)安装所需依赖包。

pip install -r requirements.txt

(6)根据自己的GPU型号,修改lib/make.sh文件中的CUDA_ARCH。
(7)由于在NMS与RoI Pooling中使用了CUDA进行加速,因此还需要对此进行编译。

cd lib && sh make.sh

笔者使用的是Pytorch 1.2.0，按照书中准备工作的流程在sh make.sh对CUDA进行编译时,报以下错误：

Compiling nms kernels by nvcc...
Traceback (most recent call last):
  File "build.py", line 4, in <module>
    from torch.utils.ffi import create_extension
  File "/home/dengjie/anaconda3/envs/pytorch/lib/python3.7/site-packages/torch/utils/ffi/__init__.py", line 1, in <module>
    raise ImportError("torch.utils.ffi is deprecated. Please use cpp extensions instead.")
ImportError: torch.utils.ffi is deprecated. Please use cpp extensions instead.
Compiling roi pooling kernels by nvcc...
Traceback (most recent call last):
  File "build.py", line 4, in <module>
    from torch.utils.ffi import create_extension
  File "/home/dengjie/anaconda3/envs/pytorch/lib/python3.7/site-packages/torch/utils/ffi/__init__.py", line 1, in <module>
    raise ImportError("torch.utils.ffi is deprecated. Please use cpp extensions instead.")
ImportError: torch.utils.ffi is deprecated. Please use cpp extensions instead.
Compiling roi align kernels by nvcc...
Traceback (most recent call last):
  File "build.py", line 4, in <module>
    from torch.utils.ffi import create_extension
  File "/home/dengjie/anaconda3/envs/pytorch/lib/python3.7/site-packages/torch/utils/ffi/__init__.py", line 1, in <module>
    raise ImportError("torch.utils.ffi is deprecated. Please use cpp extensions instead.")
ImportError: torch.utils.ffi is deprecated. Please use cpp extensions instead.
Compiling roi crop kernels by nvcc...
Traceback (most recent call last):
  File "build.py", line 4, in <module>
    from torch.utils.ffi import create_extension
  File "/home/dengjie/anaconda3/envs/pytorch/lib/python3.7/site-packages/torch/utils/ffi/__init__.py", line 1, in <module>
    raise ImportError("torch.utils.ffi is deprecated. Please use cpp extensions instead.")

这里首先参考了https://blog.csdn.net/ShuqiaoS/article/details/88420326，修改后又报了其他错误，如下：

Traceback (most recent call last):
  File "build.py", line 35, in <module>
    extra_objects=extra_objects
  File "/home/dengjie/anaconda3/envs/pytorch/lib/python3.7/site-packages/torch/utils/cpp_extension.py", line 233, in __init__
    super(BuildExtension, self).__init__(*args, **kwargs)
  File "/home/dengjie/anaconda3/envs/pytorch/lib/python3.7/site-packages/setuptools/__init__.py", line 163, in __init__
    _Command.__init__(self, dist)
  File "/home/dengjie/anaconda3/envs/pytorch/lib/python3.7/distutils/cmd.py", line 57, in __init__
    raise TypeError("dist must be a Distribution instance")

后来才发现书中的pytorch版本是0.4，而我用的是1.2.0，笔者并不想回退pytorch版本，于是到原github地址寻找答案，也就是https://github.com/jwyang/faster-rcnn.pytorch。
这个仓库里更新了对PyTorch1.0的支持，需要切换到pytorch1.0分支，具体在README.md中有写。切换分支后按照README.md中的流程，在编译CUDA依赖项时使用以下命令：

cd lib
python setup.py build develop

这时又报错：

python setup.py build developusage: setup.py [global_opts] cmd1 [cmd1_opts] [cmd2 [cmd2_opts] ...]
   or: setup.py --help [cmd1 cmd2 ...]
   or: setup.py --help-commands
   or: setup.py cmd --help

error: invalid command 'develop'

在repo中查询issue后找到解决办法
链接为https://github.com/django-extensions/django-extensions/issues/92#issuecomment-946641
具体如下：

将setup.py中的
from distutils.core import setup
修改为
from setuptools import setup

编译成功：

creating faster_rcnn.egg-info
writing faster_rcnn.egg-info/PKG-INFO
writing dependency_links to faster_rcnn.egg-info/dependency_links.txt
writing top-level names to faster_rcnn.egg-info/top_level.txt
writing manifest file 'faster_rcnn.egg-info/SOURCES.txt'
reading manifest file 'faster_rcnn.egg-info/SOURCES.txt'
writing manifest file 'faster_rcnn.egg-info/SOURCES.txt'
running build_ext
Creating /home/dengjie/anaconda3/envs/pytorch/lib/python3.7/site-packages/faster-rcnn.egg-link (link to .)
Adding faster-rcnn 0.0.0 to easy-install.pth file

Installed /home/dengjie/dengjie/Detection-PyTorch-Notebook/chapter4/faster-rcnn-pytorch-1.0/faster-rcnn.pytorch/lib
Processing dependencies for faster-rcnn==0.0.0

或者使用以下写法：

try:
    from setuptools import setup
except ImportError:
    from distutils.core import setup

以上完成对Faster RCNN PyTorch运行的准备工作，代码运行之后再更新。