Domain adaptive faster-RCNN github 复现笔记

1.项目代码链接

链接: da-faster-rcnn-Pytorch.

2.环境配置（很重要）

2.1.概述

DA-fasterrcnn的复现的主要工作在环境的配置，目前大部分的和Domain adaptive detection相关的paper和github大部分都是基于pytorch0.4.0（主要）和pytorch0.4.1两个老版本进行复现的，因此，对于显卡，CUDA，cudnn的对应配置不能使用太高或者太新的版本。
原始配置：RTX3090，CUDA11.1，cudnn8.0.5，python3.7（未复现成功）
租用服务器配置：1080ti，CUDA9.0，cudnn7.6.5，python3.6（复现成功）
另外，这里还要注意不同的，要满足CUDA版本和对应显卡的驱动的版本之间的关系，可以查看下图：
参考: CUDA版本与显卡驱动版本的对应关系.

2.2.细节

2.2.1.更新Ubuntu18.04源

①备份原始文件

cp /etc/apt/sources.list /etc/apt/sources.list.bak

②添加源

sudo gedit /etc/apt/sources.list

添加阿里源

deb http://mirrors.aliyun.com/ubuntu/ bionic main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ bionic-security main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ bionic-updates main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ bionic-proposed main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu/ bionic-backports main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic-security main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic-updates main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic-proposed main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu/ bionic-backports main restricted universe multiverse

添加清华源

deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic main restricted universe multiverse
deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic main restricted universe multiverse
deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic-updates main restricted universe multiverse
deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic-updates main restricted universe multiverse
deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic-backports main restricted universe multiverse
deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic-backports main restricted universe multiverse
deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic-security main restricted universe multiverse
deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic-security main restricted universe multiverse
deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic-proposed main restricted universe multiverse
deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic-proposed main restricted universe multiverse

添加中科大源

deb https://mirrors.ustc.edu.cn/ubuntu/ bionic main restricted universe multiverse
deb-src https://mirrors.ustc.edu.cn/ubuntu/ bionic main restricted universe multiverse
deb https://mirrors.ustc.edu.cn/ubuntu/ bionic-updates main restricted universe multiverse
deb-src https://mirrors.ustc.edu.cn/ubuntu/ bionic-updates main restricted universe multiverse
deb https://mirrors.ustc.edu.cn/ubuntu/ bionic-backports main restricted universe multiverse
deb-src https://mirrors.ustc.edu.cn/ubuntu/ bionic-backports main restricted universe multiverse
deb https://mirrors.ustc.edu.cn/ubuntu/ bionic-security main restricted universe multiverse
deb-src https://mirrors.ustc.edu.cn/ubuntu/ bionic-security main restricted universe multiverse
deb https://mirrors.ustc.edu.cn/ubuntu/ bionic-proposed main restricted universe multiverse
deb-src https://mirrors.ustc.edu.cn/ubuntu/ bionic-proposed main restricted universe multiverse

添加163源

deb http://mirrors.163.com/ubuntu/ bionic main restricted universe multiverse
deb http://mirrors.163.com/ubuntu/ bionic-security main restricted universe multiverse
deb http://mirrors.163.com/ubuntu/ bionic-updates main restricted universe multiverse
deb http://mirrors.163.com/ubuntu/ bionic-proposed main restricted universe multiverse
deb http://mirrors.163.com/ubuntu/ bionic-backports main restricted universe multiverse
deb-src http://mirrors.163.com/ubuntu/ bionic main restricted universe multiverse
deb-src http://mirrors.163.com/ubuntu/ bionic-security main restricted universe multiverse
deb-src http://mirrors.163.com/ubuntu/ bionic-updates main restricted universe multiverse
deb-src http://mirrors.163.com/ubuntu/ bionic-proposed main restricted universe multiverse
deb-src http://mirrors.163.com/ubuntu/ bionic-backports main restricted universe multiverse

2.2.2.gcc/g++降级

ubuntu18.04操作系统自带7版本gcc/g++，cuda9.0编译器要求6及其以下版本，因此需要将降级gcc/g++版本。

①查看现有gcc/g++版本

cd /usr/bin
ll gcc g++

图来源：Ubuntu18.04安装nVIDIA显卡驱动+cuda9.0+cuDNN7

②安装6版本gcc/g++

sudo apt install gcc-6 g++-6

③降级gcc/g++

cd /usr/bin
#删除软连接
sudo rm gcc
#重新建立指向gcc-6的gcc软链接
sudo ln -s gcc-6 gcc
sudo rm g++
sudo ln -s g++-6 g++

④查看gcc/g++

ll gcc g++

2.2.3.下载安装查看的CUDA9.0，pytorch0.4.0，cudnn7.6.5

2.2.3.1 CUDA

①安装CUDA

#安装CUDA-9.0
sh cuda_9.0.176_384.81_linux.run --override

②安装补丁包

sh cuda_9.0.176.1_linux.run
sh cuda_9.0.176.2_linux.run
sh cuda_9.0.176.3_linux.run
sh cuda_9.0.176.4_linux.run

③配置环境变量

gedit ~/.bashrc

④在~/.bashrc末尾添加以下内容

#cuda9.0
expert PATH=/usr/local/cuda-9.0/bin:$PATH
expert LD_LIBRARY_PATH=/usr/local/cuda-9.0/lib64:$LD_LIBRARY_PATH

⑤保存后关闭文件，生效配置文件

source ~/.bashrc

参考：Ubuntu18.04安装nVIDIA显卡驱动+cuda9.0+cuDNN7

⑥查看CUDA

nvcc -V

⑦多版本CUDA切换
首先将~/.bashrc下与cuda相关的路径均修改为/usr/local/cuda，而不使用/usr/local/cuda-9.0或者/usr/local/cuda-10.2

#删除之前在安装cuda时建立的软连接
sudo rm -rf /usr/local/cuda/

#建立新的软连接
sudo ln -s /usr/local/cuda-9.0/ /usr/local/cuda/

#查看当前版本CUDA
nvcc -V

参考：安装多版本 cuda ，多版本之间切换.

2.2.3.2 cudnn

①下载cudnn
cudnn的版本必须要和pytorch版本，pytorch对应的cudnn版本在安装pytorch时可以看到：

可以到 cudnn 下载。

②安装cudnn

#解压cudnn
tar -xzvf cudnn-9.0-linux-x64-v7.6.5.32.tgz

#复制cudnn到cuda
sudo cp cuda/include/cudnn.h /usr/local/cuda/include
sudo cp cuda/lib64/libcudnn* /usr/local/cuda/lib64
sudo chmod a+r /usr/local/cuda/include/cudnn.h /usr/local/cuda/lib64/libcudnn*

③查看cudnn

cat /usr/local/cuda/include/cudnn.h | grep CUDNN_MAJOR -A 2

④卸载cudnn

sudo rm -rf /usr/local/cuda/include/cudnn.h
sudo rm -rf /usr/local/cuda/lib64/libcudnn*

2.2.2.3 Pytorch

①安装pytorch

conda install pytorch=0.4.0
#自动安装对应版本的torchvison
codna install torchvision

②查看pytorch

conda list

3. DEBUG

3.1.安装所需python模块

pip install -r requirements.txt

3.2.下载数据集，预训练权重文件

数据集：cityscapes.
预训练权重： VGG16.链接: ResNet101.

da-faster-rcnn-pytorch
--data
  --cityscapes
  --pretrained-model
    --vgg16_caffe.pth
    --resnet101_caffe.pth

3.3.配置faster-rcnn.pytorch

cd lib
sh make.sh

3.4.常见报错及其解决方法

报错1：

TypeError:load() missing 1 required positional argument:'Loader'

解决方法：

#降级pyyaml到5.4.1
pip install pyyaml==5.4.1

报错2：

PermissionError:[Errno 13] Permission denied: ' /data'

解决方法：

进入da_trainval_net.py
line 69:
原始代码：
parser.add_argument('--save_dir', dest='save_dir',
                      help='directory to save models', default="/data/ztc/adaptation/Experiment/da_model",type=str)

修改代码：
parser.add_argument('--save_dir', dest='save_dir',
                      help='directory to save models', default="./data",type=str)

报错3：

RuntimeError: cuda runtime error (38) : no CUDA-capable device is detected at /opt/conda/conda-bld/pytorch_1535491974311/work/aten/src/THC/THCGeneral.cpp:74

报错4：

NameError: name 'nms_gpu' is not defined

解决方法:

#为CUDA,cudnn指定GPU
os.environ['CDUA_DEVICE_ORDER'] = 'PCI_BUS_ID'   #按照PCI_BUS_ID顺序从0开始排列GPU设备
os.environ['CUDA_VISIBLE_DEVICES'] = '0'  #设置当前使用的GPU设备仅为0号设备  设备名称为'/gpu:0'

注意：由于python的编译顺序是从上到下，因此，为了保证程序能顺利的利用GPU进行训练，应将指定GPU的模块放置在torch等需要使用cuda模块的前面。

报错4：

FileNotFoundError: [Errno 2] No such file or directory: '/data/ztc/detectionModel/vgg16_caffe.pth'

解决方法（resnet101_caffe.pth同理）：

cd da-faster-rcnn-Pytorch/lib/model/da_faster_rcnn

进入vgg16.py

line 21:
原始代码：
self.model_path = '/data/ztc/detectionModel/vgg16_caffe.pth'

修改代码：
self.model_path = '/data/pretrained_model/detectionModel/vgg16_caffe.pth'#修改预训练权重路径