Ubuntu16.04+python-faster-rcnn+gpu环境配置过程以及遇到的问题
参考网上很多大神的博客配置好了faster_rcnn，但配置过程比较艰难，本文主要记录一下自己的配置过程和在配置过程中遇到的各种问题。希望像博主一样刚接触faster-rcnn的新手在配置的时候少走一些弯路。
编译环境
ubuntu16.04（注：必须是真机系统，虚拟机使用的是一块虚拟显卡，不支持gpu编译，只能通过cpu训练识别，但其速度非常慢，两者速度相差约100倍。）
cuda9.0+cudnn7.3.0，cuda和cudnn的版本一定要对应安装，否则会出现不兼容的问题，下面会详细说明。
opencv3.4.9+python2.7，因为要用到ROS-kenitic，而这个版本的ROS对python2.7的适配性最好，python3的安装也大同小异。
安装ubuntu16.04双系统
安装步骤参考Win10和Ubuntu16.04双系统安装详解
ubuntu16.04镜像下载推荐使用国内镜像地址：
中科大开源镜像站
阿里云开源镜像站
安装NVIDIA显卡驱动
1.根据自己的显卡到nvidia官网查询自己的显卡驱动文件，然后下载以.run结尾的文件，并把该文件复制到home目录下，文件全路径看起来类似这样：/home/spc/NVIDIA-Linux-x86_64-440.82.run
ubuntu 16.04的默认显卡驱动是nouveau，属于第三方为nvidia显卡开发的开源驱动，要安装nvidia官方显卡驱动必须先禁用nouveau，否则会碰到冲突的问题，导致无法安装nvidia显卡驱动。
2.将ubuntu默认的显卡驱动加入黑名单
spc@ros:~$ sudo gedit /etc/modprobe.d/blacklist.conf
文件末尾添加：blacklist nouveau
3.更新系统
spc@ros:~$ sudo update-initramfs -u
4.重启系统
spc@ros:~$ reboot
5.安装nvidia驱动
-
进入文字界面
ubuntu进入文字界面：Ctrl + Alt+ F1~F6
ubuntu切换图形界面：Ctrl + Alt+ F7 -
关闭图形界面，不执行可能出错
spc@ros:~$ sudo service lightdm stop -
如果安装过其他驱动则卸载，没有可忽略
spc@ros:~$ sudo apt-get remove nvidia-* -
进入run文件所在目录
spc@ros:~$ sudo ./home/spc/NVIDIA-Linux-x86_64-440.82.run -no-x-check -no-nouveau-check -no-opengl-files-no-x-check：安装驱动时关闭X服务
-no-nouveau-check：安装驱动时禁用nouveau
-no-opengl-files：只安装驱动文件，不安装OpenGL文件
按照提示安装，安装完成后重启，若显示显卡信息则安装成功。
spc@ros:~$ sudo nvidia-smi
[sudo] spc 的密码：
Sun May 3 15:28:55 2020
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 440.82 Driver Version: 440.82 |
|-------------------------------+----------------------+----------------------+
| GPU Name Persistence-M| Bus-Id Disp.A | Volatile Uncorr. ECC |
| Fan Temp Perf Pwr:Usage/Cap| Memory-Usage | GPU-Util Compute M. |
|===============================+======================+======================|
| 0 GeForce GTX 960 Off | 00000000:01:00.0 On | N/A |
| 31% 37C P8 9W / 120W | 175MiB / 2000MiB | 0% Default |
+-------------------------------+----------------------+----------------------+
+-----------------------------------------------------------------------------+
| Processes: GPU Memory |
| GPU PID Type Process name Usage |
|=============================================================================|
| 0 942 G /usr/lib/xorg/Xorg 158MiB |
| 0 1565 G compiz 4MiB |
| 0 1876 G fcitx-qimpanel 6MiB |
| 0 2877 G /usr/lib/firefox/firefox 0MiB |
+-----------------------------------------------------------------------------+
安装cuda
CUDA是NVIDIA的编程语言平台，从CUDA 9版本开始，Ubuntu 16.04，Ubuntu 17.04和Fedora 25完全支持gcc 6，Ubuntu16.04自带gcc 5.4，所以不能安装cuda8.0，应该安装cuda9.0以上版本，否则，需要将gcc版本降低。
- 安装依赖库
spc@ros:~$ sudo apt-get install freeglut3-dev build-essential libx11-dev libxmu-dev libxi-dev libgl1-mesa-glx libglu1-mesa libglu1-mesa-dev
- 从cuda9.0官网下载run文件
- 将文件放到home目录下安装
spc@ros:~$ sudo chmod 777 cuda_9.0.176_384.81_linux.run
spc@ros:~$ sudo ./cuda_9.0.176_384.81_linux.run
*注：按照提示完成安装，安装路径都直接默认即可，在选择是否选择安装nvidia驱动时选择否
- 环境变量配置
打开~/.bashrc文件：
spc@ros:~$ sudo gedit ~/.bashrc
在文件末尾加入以下内容：
export PATH=/usr/local/cuda-9.0/bin${PATH:+:${PATH}}
export LD_LIBRARY_PATH=/usr/local/cuda-9.0/lib64${LD_LIBRARY_PATH:+:${LD_LIBRARY_PATH}}
- 测试CUDA的例子
spc@ros:~$ cd /usr/local/cuda-9.0/samples/1_Utilities/deviceQuery
spc@ros:/usr/local/cuda-9.0/samples/1_Utilities/deviceQuery$ sudo make
spc@ros:/usr/local/cuda-9.0/samples/1_Utilities/deviceQuery$ ./deviceQuery
//若显示Result = PASS则安装成功
安装opencv
到opencv官网下载source，题主使用opencv3.3.0+opencv-contrib-3.3.0编译的时候出现Build output check failed的错误参照网上大神的解决办法始终没有效果，然后改用opencv3.4.9版本，成功编译。进入opencv所在目录编译：
cd opencv-3.4.9
mkdir build && cd build/
sudo cmake -D CMAKE_BUILD_TYPE=Release -D CMAKE_INSTALL_PREFIX=/usr/local ..
sudo make -j8
sudo make install
//j8表示开启多个线程编译，根据自己的电脑配置设置
#error "C++11 is not supported"的问题
ubuntu16.04 g++升级过程
详细参考环境配置—Ubuntu 16.04 安装Opencv 3.4.3
编译py-faster-rcnn
- 下载源码
git clone --recursive https://github.com/rbgirshick/py-faster-rcnn.git
注：如果执行完git clone指令后发现 py-faster-rcnn/caffe-fast-rcnn目录是空的，则需要继续下载，执行如下指令：
cd caffe-fast-rcnn
git submodule update --init --recursive
- 由于caffe-faster-rcnn中对高于cuda4.0版本不兼容的问题，所以下载最新的caffe-master
用新版caffe-master中的文件替换caffe-faster-rcnn中的文件，指令如下：
cp caffe/include/caffe/layers/cudnn_* py-faster-rcnn/caffe-fast-rcnn/include/caffe/layers/
cp caffe/src/caffe/layers/cudnn_* py-faster-rcnn/caffe-fast-rcnn/src/caffe/layers/
cp caffe/include/caffe/util/cudnn.hpp py-faster-rcnn/caffe-fast-rcnn/include/caffe/util/
- 进入caffe-faster-rcnn目录，创建Makefile.config文件，复制下面内容：
## Refer to http://caffe.berkeleyvision.org/installation.html
# Contributions simplifying and improving our build system are welcome!
# cuDNN acceleration switch (uncomment to build with cuDNN).
USE_CUDNN := 1
# CPU-only switch (uncomment to build without GPU support).
# CPU_ONLY := 1
# uncomment to disable IO dependencies and corresponding data layers
# USE_OPENCV := 0
# USE_LEVELDB := 0
# USE_LMDB := 0
# uncomment to allow MDB_NOLOCK when reading LMDB files (only if necessary)
# You should not set this flag if you will be reading LMDBs with any
# possibility of simultaneous read and write
# ALLOW_LMDB_NOLOCK := 1
# Uncomment if you're using OpenCV 3
OPENCV_VERSION := 3
# To customize your choice of compiler, uncomment and set the following.
# N.B. the default for Linux is g++ and the default for OSX is clang++
# CUSTOM_CXX := g++
# CUDA directory contains bin/ and lib/ directories that we need.
CUDA_DIR := /usr/local/cuda
# On Ubuntu 14.04, if cuda tools are installed via
# "sudo apt-get install nvidia-cuda-toolkit" then use this instead:
# CUDA_DIR := /usr
# CUDA architecture setting: going with all of them.
# For CUDA < 6.0, comment the *_50 lines for compatibility.
CUDA_ARCH := -gencode arch=compute_30,code=sm_30 \
-gencode arch=compute_35,code=sm_35 \
-gencode arch=compute_50,code=sm_50 \
-gencode arch=compute_50,code=compute_50
# BLAS choice:
# atlas for ATLAS (default)
# mkl for MKL
# open for OpenBlas
BLAS := atlas
# Custom (MKL/ATLAS/OpenBLAS) include and lib directories.
# Leave commented to accept the defaults for your choice of BLAS
# (which should work)!
# BLAS_INCLUDE := /path/to/your/blas
# BLAS_LIB := /path/to/your/blas
# Homebrew puts openblas in a directory that is not on the standard search path
# BLAS_INCLUDE := $(shell brew --prefix openblas)/include
# BLAS_LIB := $(shell brew --prefix openblas)/lib
# This is required only if you will compile the matlab interface.
# MATLAB directory should contain the mex binary in /bin.
# MATLAB_DIR := /usr/local
# MATLAB_DIR := /Applications/MATLAB_R2012b.app
# NOTE: this is required only if you will compile the python interface.
# We need to be able to find Python.h and numpy/arrayobject.h.
PYTHON_INCLUDE := /usr/include/python2.7 \
/usr/lib/python2.7/dist-packages/numpy/core/include
# Anaconda Python distribution is quite popular. Include path:
# Verify anaconda location, sometimes it's in root.
# ANACONDA_HOME := $(HOME)/anaconda
# PYTHON_INCLUDE := $(ANACONDA_HOME)/include \
# $(ANACONDA_HOME)/include/python2.7 \
# $(ANACONDA_HOME)/lib/python2.7/site-packages/numpy/core/include \
# Uncomment to use Python 3 (default is Python 2)
# PYTHON_LIBRARIES := boost_python3 python3.5m
# PYTHON_INCLUDE := /usr/include/python3.5m \
# /usr/lib/python3.5/dist-packages/numpy/core/include
# We need to be able to find libpythonX.X.so or .dylib.
PYTHON_LIB := /usr/lib
# PYTHON_LIB := $(ANACONDA_HOME)/lib
# Homebrew installs numpy in a non standard path (keg only)
# PYTHON_INCLUDE += $(dir $(shell python -c 'import numpy.core; print(numpy.core.__file__)'))/include
# PYTHON_LIB += $(shell brew --prefix numpy)/lib
# Uncomment to support layers written in Python (will link against Python libs)
WITH_PYTHON_LAYER := 1
# Whatever else you find you need goes here.
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib /usr/lib/x86_64-linux-gnu /usr/lib/x86_64-linux-gnu/hdf5/serial
# If Homebrew is installed at a non standard location (for example your home directory) and you use it for general dependencies
# INCLUDE_DIRS += $(shell brew --prefix)/include
# LIBRARY_DIRS += $(shell brew --prefix)/lib
# Uncomment to use `pkg-config` to specify OpenCV library paths.
# (Usually not necessary -- OpenCV libraries are normally installed in one of the above $LIBRARY_DIRS.)
# USE_PKG_CONFIG := 1
BUILD_DIR := build
DISTRIBUTE_DIR := distribute
# Uncomment for debugging. Does not work on OSX due to https://github.com/BVLC/caffe/issues/171
# DEBUG := 1
# The ID of the GPU that 'make runtest' will use to run unit tests.
TEST_GPUID := 0
# enable pretty build (comment to see full commands)
Q ?= @
- 编译：make -j8 && make pycaffe
至此，可以使用faster-rcnn了，感谢大神们的问题记录和解决方法。贴一个编译成功的过程，比cpu
快了100倍。
Loaded network /home/spc/py-faster-rcnn/data/faster_rcnn_models/VGG16_faster_rcnn_final.caffemodel
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Demo for data/demo/000456.jpg
Detection took 0.250s for 300 object proposals
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Demo for data/demo/000542.jpg
Detection took 0.214s for 161 object proposals
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Demo for data/demo/001150.jpg
Detection took 0.220s for 194 object proposals
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Demo for data/demo/001763.jpg
Detection took 0.219s for 196 object proposals
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Demo for data/demo/004545.jpg
Detection took 0.264s for 300 object proposals
问题总结
make[2]: *** No rule to make target ‘/usr/lib/x86_64-linux-gnu/libGL.so’
ImportError: ‘No module named skimage.io’
ImportError: No module named ‘yaml’
ValueError: numpy.ufunc size changed, may indicate binary incompatibility.
注:这个问题如果是python2.7产生的，需要先升级pip，才能升级numpy
importError: No module named easydict
如遇到g++/gcc版本太高则切换版本后再编译
linux下gcc、g++不同版本的安装和切换
1459
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
