记录Ubuntu18.04-cuda10.1-opencv4配置caffe过程

注意1：caffe和较高版本的cuda兼容不好，建议使用cuda-10.1以下的版本；
注意2：编译的选择gcc8以下的版本；
注意3：对于pycaffe的编译，如果用系统的Python，cmake的时候需要先把anaconda的环境deactivate，否则版本不匹配，建议用系统的Python，无论Python3还是Python2。对于Python3和Python2的选择，如果Python水平还可以，当然是Python3优先，还Python2上修改比较方便，否则，最好按照目标项目来，因为caffe比较古老，所以对应的项目一般也比较古老，多数都是Python2；
注意4：关于boost_python3的问题，如果是Ubuntu20，需要到/usr/lib/x86_64-linux-gnu目录下看一下你的boostpython3是那个版本和库的具体文件名字是什么，例如：

此时要将PYTHON_LIBRARIES ?= boost_python3 python3.8，改为：PYTHON_LIBRARIES ?= boost_python38 python3.8

• 关于修改相关软连接的方法，参考（不需要删除高版本）：
  https://blog.csdn.net/sunmingyang1987/article/details/125515341

---

1、安装前环境

cuda10.1
Ubuntu18.04
Python==3.7
opencv4

2、准备工作

2.1 必要依赖

sudo apt-get install libprotobuf-dev libleveldb-dev libsnappy-dev libopencv-dev libhdf5-serial-dev protobuf-compiler
 
sudo apt-get install --no-install-recommends libboost-all-dev
 
sudo apt-get install libopenblas-dev liblapack-dev libatlas-base-dev
 
sudo apt-get install libgflags-dev libgoogle-glog-dev liblmdb-dev
 
sudo apt-get install git cmake build-essential

2.2 安装python3-numpy

sudo apt install python3-numpy

2.3 安装HDF5

请参考这篇博客：HDF5安装及问题解决
安装过程很顺利！
注意：这一步不安装可能导致后面出现以下错误：

.build_release/tools/caffe: error while loading shared libraries: libhdf5_hl.so.100: cannot open shared object file: No such file or directory
Makefile:526: recipe for target 'runtest' failed

解决方法是：从’anaconda2/lib’下面找到相应的文件复制到’/usr/lib/x86_64-linux-gnu’中即可。

3、安装caffe

3.1 下载caffe

git clone git://github.com/BVLC/caffe.git
 
cd caffe/
 
cp Makefile.config.example Makefile.config

3.2 编辑Makefile.config

3.2.1 取消下面几行注释

取消注释#USE_CUDNN :=1
取消注释#OPENCV—VERSION :=3
取消注释#WITH_PYTHON_LAYER := 1

3.2.2 修改Python版本

将系统自带的Python2改为希望的Python版本，把原来Python2.7的接口注释掉，然后添加相应Python版本的接口。

# NOTE: this is required only if you will compile the python interface.
# We need to be able to find Python.h and numpy/arrayobject.h.
# PYTHON_INCLUDE := /usr/include/python2.7 \
#		/usr/lib/python2.7/dist-packages/numpy/core/include
# Anaconda Python distribution is quite popular. Include path:
# Verify anaconda location, sometimes it's in root.
# ANACONDA_HOME := $(HOME)/anaconda
# PYTHON_INCLUDE := $(ANACONDA_HOME)/include \
		# $(ANACONDA_HOME)/include/python2.7 \
		# $(ANACONDA_HOME)/lib/python2.7/site-packages/numpy/core/include

# Uncomment to use Python 3 (default is Python 2) 
PYTHON_LIBRARIES := boost_python3 python3.7
PYTHON_INCLUDE := /usr/include/python3.7 \
                 /usr/lib/python3.7/dist-packages/numpy/core/include

3.2.3 CUDA_ARCH修改

删除CUDA_ARCH的两个值

:-gencode arch=compute_20,code=sm_20 \
     -gencode arch=compute_20,code=sm_21 \

3.2.4 将以下几行注释去掉，并修改值为1

# uncomment to disable IO dependencies and corresponding data layers
USE_OPENCV := 1
USE_LEVELDB := 1
USE_LMDB := 1
# This code is taken from https://github.com/sh1r0/caffe-android-lib
USE_HDF5 := 1

3.2.5 HDF5相关修改

这里主要是对下面两行进行修改，将hdf5相关路径添加上即可：

INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include/ /usr/include/hdf5/serial
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib  /usr/local/hdf5/lib /usr/lib/x86_64-linux-gnu

3.2.6 opencv4相关的修改

首先修改config文件中的内容，添加OpenCV4的目录:
这里我分别添加了/usr/local/opencv/include/opencv4/和/usr/local/opencv/lib两个路径，根据自己的安装情况添加。

INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include/ /usr/include/hdf5/serial /usr/local/include/opencv4/
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib //usr/local/hdf5/lib /usr/lib/x86_64-linux-gnu /usr/local/opencv/lib

INCLUDE_DIRS中添加的OpenCV目录是包含’opencv2’文件夹的目录

然后，再修改cpp文件中某些变量名称，因为OpenCV3和OpenCV4之间有些稍微不同，在caffe安装目录下输入：

sed -i 's/CV_LOAD_IMAGE_COLOR/cv::IMREAD_COLOR/g' src/caffe/layers/window_data_layer.cpp
sed -i 's/CV_LOAD_IMAGE_COLOR/cv::IMREAD_COLOR/g' src/caffe/util/io.cpp
sed -i 's/CV_LOAD_IMAGE_COLOR/cv::IMREAD_COLOR/g' src/caffe/test/test_io.cpp
sed -i 's/CV_LOAD_IMAGE_GRAYSCALE/cv::ImreadModes::IMREAD_GRAYSCALE/g' src/caffe/util/io.cpp
sed -i 's/CV_LOAD_IMAGE_GRAYSCALE/cv::ImreadModes::IMREAD_GRAYSCALE/g' src/caffe/test/test_io.cpp

3.3 修改Makefile文件

找到这一行，并修改为以下形式：

LIBRARIES += glog gflags protobuf boost_system boost_filesystem m hdf5_serial_hl hdf5_serial

3.4 进行编译

sudo make all -j11 #我用了11个核

编译过程顺利结束！

4、测试

先后执行以下命令：

sudo make test -j10

make runtest -j10

通过测试！！！

5、编译pycaffe

执行：

sudo make pycaffe -j10

修改Python路径，让Python可以找到caffe模块：

sudo vim  ~/.bashrc
export PYTHONPATH={Your caffe path}/python:$PYTHONPATH
source ~/.bashrc

进入IPython测试：

安装完成。

后记：

python 的caffe api接口安装后在使用时会提示有些模块找不到
例如：ImportError: No module named ‘google’，解决方法如下：
pip install --upgrade google-api-python-client

然后又报错：

TypeError: Couldn't build proto file into descriptor pool: duplicate file name (caffe/proto/caffe.proto)

这个问题在网上也能找到答案：https://blog.csdn.net/qq_38963393/article/details/124299998，解决办法：

reason:
	this is an issue with protobuf binary wheels.
solution:
	The work around it to install the pure python implementation or fix proto files,
	so they don't have namespace collisions.

install pure python implementation:
	pip uninstall protobuf
	pip install --no-binary=protobuf protobuf

Cmake编译

mkdir build
cd build
cmake ..   

上面一步完了，不要cd到根目录，直接在build目录下make

sudo make all -j12
sudo make runtest -j12
sudo make pycaffe -j12

建议cmake编译，更简单，不易出错，注意选择对应的Python版本：
pycaffe安装完后，会建立build的_caffe.so库的软连接到caffe根目录下的Python文件夹内，所以PYTHONPATH的添加方式和之前一样。