Caffe搭建手记_基于Ubuntu14.04LTS

原创作品，请勿转载！
博客源地址：http://blog.csdn.net/dawin_2008/article/details/52382237

CAFFE+Ubuntu14.04LTS+CUDA7.5+cuDNN+OpenCV3.0+OpenBLAS+Anaconda+Matlab R2014a

我的硬件配置：
内存：4GB
处理器：i3-2120CPU@3.30GHz * 4
图形：GeForce GTX 960/PCIe/SSE2
操作系统：Ubuntu14.04LTS 64位
磁盘：100GB

注意以下几点：

1、一定要在root环境下安装，否则出现很多不必要的授权问题，例如不能读lib
2、在root环境下修改~/.bashrc，否则出现lib文件路径不存在的问题
3、在终端安装第三方文件要测试网速，否则有些会报错，例如opencv有一个第三方插件，我会把要安装的放在csdn下载地址，供大家下载

Ubuntu14.04LTS安装

我是采用双系统的，在Windows7环境下安装easyBCD，然后按照百度经验的流程走就行了，参考如下：

http://jingyan.baidu.com/article/e4d08ffdace06e0fd2f60d39.html

安装依赖库

sudo apt-get install build-essential # basic requirement
sudo apt-get install libprotobuf-dev libleveldb-dev libsnappy-dev libopencv-dev libboost-all-dev libhdf5-serial-dev libgflags-dev libgoogle-glog-dev liblmdb-dev protobuf-compiler #required by caffe
sudo apt-get install libgflags-dev libgoogle-glog-dev liblmdb-dev


CUDA Toolkit 7.5安装

1.查看自己的显卡是否能安装CUDA Toolkit 7.5

$ lspci | grep -i nvidia

只要型号存在于https://developer.nvidia.com/cuda-gpus，就没问题了

2.下载 NVIDIA CUDA Toolkit7.5

下载地址：https://developer.nvidia.com/cuda-toolkit
$ md5sum <filename>
验证地址：https://developer.nvidia.com/rdp/cuda-rc-checksums

我下载的是 cuda-repo-ubuntu1404-7-5-local_7.5-18_amd64.deb，关闭lightdm，以免和原来的驱动发生冲突：
$ sudo service lightdm stop
按下ctrl+alt+F2进入命令行模式，在命令行中输入：
$ cd /path/to/Downloads
$ sudo dpkg -i cuda-repo-ubuntu1404-7-5-local_7.5-18_amd64.deb
$ sudo apt-get update
$ sudo apt-get install -y cuda

添加环境变量到系统配置文件profile：
# sudo gedit /etc/profile
在最后两行添加
export PATH=/usr/local/cuda-7.5/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda-7.5/lib64:$LD_LIBRARY_PATH
安装完成，重启。

3.安装NVIDIA cuDNN（可选，是用于深度神经网络的GPU加速库）
下载cuDNN，从https://developer.nvidia.com/rdp/cudnn-download，注册，然后下载，我下载的是cudnn-7.0-linux-x64-v4.0-prod.tgz:
$ sudo tar xvf cudnn-7.0-linux-x64-v4.0-prod.tgz
$ cd cuda/include
$ sudo cp *.h /usr/local/include/
$ cd ../lib64
$ sudo cp lib* /usr/local/lib/
$ cd /usr/local/lib
$ sudo chmod +r libcudnn.so.4.0.7
$ sudo ln -sf libcudnn.so.4.0.7 libcudnn.so.4
$ sudo ln -sf libcudnn.so.4 libcudnn.so
$ sudo ldconfig #这里有报错：/sbin/ldconfig.real: /usr/local/lib/libcudnn.so.4 is not a symbolic link但是看来不影响

设置环境变量，添加CUDA环境变量：
sudo gedit /etc/profile
在最后两行继续添加
export PATH=/usr/local/cuda/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH
保存后，使环境变量立即生效，
$ source /etc/profile
进入/usr/local/cuda/samples，执行下面的命令来build samples:
$ sudo make all -j4
全部编译完成后，进入 samples/bin/x86_64/linux/release，运行deviceQuery:
$ ./deviceQuery

还可以用$ nvcc --version查看版本号，若提示nvcc命令不存在，说明安装有错！

MATLAB R2014a安装

1.挂载iso（需新建matlab_iso文件夹）：
$ sudo mount -o loop MATHWORKS_R2014A.iso ～/matlab_iso

2.开始安装：
$ cd ~/matlab_iso
$ sudo ./install

3.选择”install manually without using the internet”项进行安装

4.输入”file installation key”:12345-67890-12345-67890（随便都行）

5.激活：选择”license_405329_R2014a.lic”文件进行激活（在Crack文件夹下面）

6.将libmwservices.so复制到/usr/local/MATLAB/R2014a/bin/glnxa64中：
$ sudo cp libmwservices.so /usr/local/MATLAB/R2014a/bin/glnxa64/libmwservices.so

7.验证：
$ matlab
若不报错就是安装成功

解决编译器gcc/g++版本问题

因为Ubuntu 14.04的gcc/g++版本是4.8.4，而Matlab R2014a（2015a）的版本是4.7.x所以在使用matla调用mex文件的时候，基本上都会报错，根据报错信息，考虑如下两步解决方案。
1.下载gcc/g++ 4.7.x
$ sudo apt-get install -y gcc-4.7
$ sudo apt-get install -y g++-4.7

2.链接gcc/g++实现降级
$ cd /usr/bin
$ sudo rm gcc
$ sudo ln -s gcc-4.7 gcc
$ sudo rm g++
$ sudo ln -s g++-4.7 g++
降级成功！

安装OpenCV3.0

下载OpenCV3.0:
OpenCV3.0
安装依赖库：
$ sudo apt-get install build-essential
$ sudo apt-get install cmake git libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev
$ sudo apt-get install python-dev python-numpy libtbb2 libtbb-dev libjpeg-dev libpng-dev libtiff-dev libjasper-dev libdc1394-22-dev
在/home/username/中新建/opencv
$ cd ~/opencv
$ mkdir release
$ cd release
$ cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local ..
$ make
$ sudo make install
编译samples程序:
$ cd ~/opencv/samples
$ sudo cmake .
$ sudo make -j $(nproc)
运行测试程序:
(测试程序根据版本不同会有不同的数据路径等等)
# 注：使用alpha版本，images在opencv/cpp文件夹下
$ cd cpp/
$ ./cpp-example-facedetect lena.jpg

安装OpenBLAS

1.下载OpenBLAS:

http://github.com/xianyi/OpenBLAS/archive/v0.2.19.tar.gz

2.安装：
$ mkdir ~/OpenBLAS
$ cd ~/OpenBLAS
$ tar zxvf OpenBLAS-0.2.19.tar.gz
$ make
$ make PREFIX=/home/username/OpenBLAS
3、添加环境变量到/etc/profile并source：
export LD_LIBRARY_PATH=/home/dawin/OpenBLAS:$LD_LIBRARY_PATH

安装Anaconda python

1.下载Anaconda

https://repo.continuum.io/archive/Anaconda2-4.1.1-Linux-x86_64.sh

2.安装：
$ bash ~/Downloads/Anaconda3-4.0.0-Linux-x86_64.sh

3.添加环境变量，不用每次都要手动输入
3.1.修改profile文件：
在里面最后一行加入:
export PATH=/home/username/anaconda2/bin:$PATH
让环境变量立即生效需要执行如下命令：
$ source /etc/profile

编译caffe

1.下载caffe-master包，解压重命名为caffe，我放在/home目录下

git@github.com:BVLC/caffe.git

2.进入caffe目录，编译主程序
$ cp Makefile.config.example Makefile.config
$ sudo gedit Makefile.config

参数说明

• CPU_ONLY 是否只使用CPU模式，没有GPU没安装CUDA的同学可以打开这个选项
• BLAS (使用intel mkl、atlas还是OpenBLAS)我用的是优化后的BLAS，即OpenBLAS，所以 BLAS=：open 并加入库文件路径
• MATLAB_DIR 如果需要使用MATLAB wrapper的同学需要指定matlab的安装路径, 如我的路径为 /usr/local/MATLAB/R2014a (注意该目录下需要包含bin文件夹，bin文件夹里应该包含mex二进制程序)

我的Makefile.config配置如下：

## Refer to http://caffe.berkeleyvision.org/installation.html
# Contributions simplifying and improving our build system are welcome!

# cuDNN acceleration switch (uncomment to build with cuDNN).
USE_CUDNN := 1

# CPU-only switch (uncomment to build without GPU support).
# CPU_ONLY := 1

# uncomment to disable IO dependencies and corresponding data layers
# USE_OPENCV := 0
# USE_LEVELDB := 0
# USE_LMDB := 0

# uncomment to allow MDB_NOLOCK when reading LMDB files (only if necessary)
#   You should not set this flag if you will be reading LMDBs with any
#   possibility of simultaneous read and write
# ALLOW_LMDB_NOLOCK := 1

# Uncomment if you're using OpenCV 3
OPENCV_VERSION := 3

# To customize your choice of compiler, uncomment and set the following.
# N.B. the default for Linux is g++ and the default for OSX is clang++
# CUSTOM_CXX := g++

# CUDA directory contains bin/ and lib/ directories that we need.
CUDA_DIR := /usr/local/cuda
# On Ubuntu 14.04, if cuda tools are installed via
# "sudo apt-get install nvidia-cuda-toolkit" then use this instead:
# CUDA_DIR := /usr

# CUDA architecture setting: going with all of them.
# For CUDA < 6.0, comment the *_50 lines for compatibility.
CUDA_ARCH := -gencode arch=compute_20,code=sm_20 \
        -gencode arch=compute_20,code=sm_21 \
        -gencode arch=compute_30,code=sm_30 \
        -gencode arch=compute_35,code=sm_35 \
        -gencode arch=compute_50,code=sm_50 \
        -gencode arch=compute_50,code=compute_50

# BLAS choice:
# atlas for ATLAS (default)
# mkl for MKL
# open for OpenBlas
BLAS := open
# Custom (MKL/ATLAS/OpenBLAS) include and lib directories.
# Leave commented to accept the defaults for your choice of BLAS
# (which should work)!
BLAS_INCLUDE := /opt/OpenBLAS/include
BLAS_LIB := /opt/OpenBLAS/lib

# Homebrew puts openblas in a directory that is not on the standard search path
# BLAS_INCLUDE := $(shell brew --prefix openblas)/include
# BLAS_LIB := $(shell brew --prefix openblas)/lib

# This is required only if you will compile the matlab interface.
# MATLAB directory should contain the mex binary in /bin.
MATLAB_DIR := /usr/local/MATLAB/R2014a
#MATLAB_DIR := /Applications/MATLAB_R2012b.app

# NOTE: this is required only if you will compile the python interface.
# We need to be able to find Python.h and numpy/arrayobject.h.
PYTHON_INCLUDE := /usr/include/python2.7 \
        /usr/lib/python2.7/dist-packages/numpy/core/include
# Anaconda Python distribution is quite popular. Include path:
# Verify anaconda location, sometimes it's in root.
ANACONDA_HOME := $(HOME)/anaconda2
PYTHON_INCLUDE := $(ANACONDA_HOME)/include \
           $(ANACONDA_HOME)/include/python2.7 \
           $(ANACONDA_HOME)/lib/python2.7/site-packages/numpy/core/include \

# Uncomment to use Python 3 (default is Python 2)
# PYTHON_LIBRARIES := boost_python3 python3.5m
# PYTHON_INCLUDE := /usr/include/python3.5m \
#                 /usr/lib/python3.5/dist-packages/numpy/core/include

# We need to be able to find libpythonX.X.so or .dylib.
PYTHON_LIB := /usr/lib
PYTHON_LIB := $(ANACONDA_HOME)/lib

# Homebrew installs numpy in a non standard path (keg only)
# PYTHON_INCLUDE += $(dir $(shell python -c 'import numpy.core; print(numpy.core.__file__)'))/include
# PYTHON_LIB += $(shell brew --prefix numpy)/lib

# Uncomment to support layers written in Python (will link against Python libs)
# WITH_PYTHON_LAYER := 1

# Whatever else you find you need goes here.
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib

# If Homebrew is installed at a non standard location (for example your home directory) and you use it for general dependencies
# INCLUDE_DIRS += $(shell brew --prefix)/include
# LIBRARY_DIRS += $(shell brew --prefix)/lib

# Uncomment to use `pkg-config` to specify OpenCV library paths.
# (Usually not necessary -- OpenCV libraries are normally installed in one of the above $LIBRARY_DIRS.)
# USE_PKG_CONFIG := 1

# N.B. both build and distribute dirs are cleared on `make clean`
BUILD_DIR := build
DISTRIBUTE_DIR := distribute

# Uncomment for debugging. Does not work on OSX due to https://github.com/BVLC/caffe/issues/171
# DEBUG := 1

# The ID of the GPU that 'make runtest' will use to run unit tests.
TEST_GPUID := 0

# enable pretty build (comment to see full commands)
Q ?= @

$ make all -j4 #使用多核加速，我的是四核
$ make test
$ make runtest

出现报错信息:

error while loading shared libraries: libhdf5_hl.so.10: cannot open shared object file: No such file or directory

凡是这种情况,如果报错文件的确是存在,就把文件所在的路径加入环境变量,在~./bashrc添加文件所在路径,这是我成功编译后的bashrc文件添加的路径:

export PATH=/usr/local/cuda/bin:$PATH
export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH
#export LD_LIBRARY_PATH=/home/dawin/anaconda2/lib:$LD_LIBRARY_PATH
#请勿添加此条到/etc/profile中，否则无法进入系统
export LD_LIBRARY_PATH=/home/dawin/OpenBLAS:$LD_LIBRARY_PATH

再make runtest,效果如图:

3.编译Matlab wrapper
$ make matcaffe
然后就可以跑官方的matlab demo!

4.编译Python wrapper
$ make pycaffe
修改profile文件：
export PYTHONPATH=/home/username/caffe/python:$PYTHONPATH
以后也可以跑官方的demo

5.验证python接口
$ python
在python命令环境下输入import caffe,出现报错:

ImportError: No module named google.protobuf.internal

这是因为有两个python路径在变量环境中,要给anaconda中python和系统中的python安装protobuf,退出python,输入以下命令:
$ sudo pip install protobuf
$ sudo /home/dawin/anaconda2/bin/pip install protobuf
再次在python命令环境下输入import caffe,不报错就成功了!

后记

安装caffe真是一个大工程,我从windows里面安装caffe,再到ubuntu中安装花了半个月,接着试图测试fast-rcnn和faster-rcnn都没成功,最后测试rcnn成功,可是前几天手贱输入nvcc,提示没有安装nvcc,就重新安装nvcc,导致显卡冲突并且不能重新安装nvidia的显卡,只能被逼着重装ubuntu,道路相当曲折,在此过程中感谢我的两位导师–贺博和周博,没有他们的指导我很难继续下去,希望这是我做神经网络相关课题的良好开端

参考网站:

http://caffe.berkeleyvision.org/installation.html#compilation
http://blog.csdn.net/YhL_Leo/article/details/50961542
http://www.cnblogs.com/wm123/p/5385940.html
http://blog.mindcont.com/2016/07/20/ubuntu1404-caffe-cuda7-5-mkl/
http://blog.csdn.net/u013915633/article/details/49886465