caffe的搭建过程以及遇上的各种问题的汇总

最新推荐文章于 2023-04-27 14:04:08 发布

huangjx07

最新推荐文章于 2023-04-27 14:04:08 发布

阅读量2.1k

点赞数

文章标签： caffe ubuntu

本文链接：https://blog.csdn.net/huangjx07/article/details/52608084

版权

Caffe的搭建过程以及遇上的各种问题的汇总：

ubuntu15.04+CUDA7.5+opencv3.0.0+python2.7

整个搭建过程参考 http://blog.csdn.net/ubunfans/article/details/47724341，过程中遇上了各种花式的问题，废了两天终于将caffe框架搭好。整理一下过程和遇到的问题，给自己长长经验。

caffe安装步骤：

（声明：从第一步到第六步跟我上面提及到的那个博客是一样的，安装过程可以自行参考。经验：除非框架有特殊要求，否则其他例如cuda,opencv,python等尽量不要装最新的版本，毕竟刚刚出来的版本后续的都没有跟上，会很大程度的加大出错率和搭建的苦难度。）

1.安装开发所需的依赖项：

sudo apt-get install build-essential  # basic requirement
sudo apt-get install libprotobuf-dev libleveldb-dev libsnappy-dev libopencv-dev libboost-all-dev
libhdf5-serial-dev libgflags-dev libgoogle-glog-dev liblmdb-dev protobuf-compiler #required by caffe

2.安装CUDA7.5:

安装cuda也很简单，到cuda官网下载你需要的版本，然后按照它提示的那几句代码安装就可以啦，建议下deb包离线安装。CUDA的deb包比较大，用ubuntu系统下载的话速度慢，建议用windows下载再拷过去（后面遇上比较大的文件下载我都是用windows下的）。。

官网和我参考的博客都提及到用md5检验下下载的deb包，这个自行选择啦~

下载到deb包后，cd到所在的目录，执行下面的语句：

sudo dpkg -i cuda-repo-
   
   
    
    _
    
    
     
     _
     
     
      
      .deb  
sudo apt-get update  
sudo apt-get install cuda

装好后最好重启一下系统再接着接下来的步骤。

3.安装cudnn

到nvidia官网下载cudnn(先注册，后下载。我下载的是cudnn-7.5-linux-x64-v5.1.tgz)

tar -zxvf cudnn-7.5-linux-x64-v5.1.tgz  
cd cuda  
sudo cp lib64/lib* /usr/local/cuda/lib64/  
sudo cp include/cudnn.h /usr/local/cuda/include/ 

#更新软链接
cd /usr/local/cuda/lib64/
sudo chmod +r libcudnn.so.5.1.3  #注意版本号，不一样记得要改
sudo ln -sf libcudnn.so.5.1.3 libcudnn.so.5
sudo ln -sf libcudnn.so.5 libcudnn.so
sudo ldconfig

4.设置环境变量

添加cuda环境变量：

sudo gedit /etc/profile

添加内容：

PATH=/usr/local/cuda/bin:$PATH
export PATH

使环境变量生效:

source /etc/profile

添加lib库路径：

sudo gedit /etc/ld.so.conf.d/cuda.conf

添加内容：

/usr/local/cuda/lib64

使路径生效:

sudo ldconfig

5.安装cuda samples

cd /usr/local/cuda/samples   #bulid samples
sudo make all -j4
#编译完成后
cd bin/x86_64/linux/release
./deviceQuery

如果前面的都安装成功，此处可以显示显卡的信息：

./deviceQuery Starting...

 CUDA Device Query (Runtime API) version (CUDART static linking)

Detected 1 CUDA Capable device(s)

Device 0: "GeForce GTX 980"
  CUDA Driver Version / Runtime Version          7.5 / 7.5
  CUDA Capability Major/Minor version number:    5.2
  Total amount of global memory:                 4093 MBytes (4291493888 bytes)
  (16) Multiprocessors, (128) CUDA Cores/MP:     2048 CUDA Cores
  GPU Max Clock rate:                            1216 MHz (1.22 GHz)
  Memory Clock rate:                             3505 Mhz
  Memory Bus Width:                              256-bit
  L2 Cache Size:                                 2097152 bytes
  Maximum Texture Dimension Size (x,y,z)         1D=(65536), 2D=(65536, 65536), 3D=(4096, 4096, 4096)
  Maximum Layered 1D Texture Size, (num) layers  1D=(16384), 2048 layers
  Maximum Layered 2D Texture Size, (num) layers  2D=(16384, 16384), 2048 layers
  Total amount of constant memory:               65536 bytes
  Total amount of shared memory per block:       49152 bytes
  Total number of registers available per block: 65536
  Warp size:                                     32
  Maximum number of threads per multiprocessor:  2048
  Maximum number of threads per block:           1024
  Max dimension size of a thread block (x,y,z): (1024, 1024, 64)
  Max dimension size of a grid size    (x,y,z): (2147483647, 65535, 65535)
  Maximum memory pitch:                          2147483647 bytes
  Texture alignment:                             512 bytes
  Concurrent copy and kernel execution:          Yes with 2 copy engine(s)
  Run time limit on kernels:                     Yes
  Integrated GPU sharing Host Memory:            No
  Support host page-locked memory mapping:       Yes
  Alignment requirement for Surfaces:            Yes
  Device has ECC support:                        Disabled
  Device supports Unified Addressing (UVA):      Yes
  Device PCI Domain ID / Bus ID / location ID:   0 / 1 / 0
  Compute Mode:
     < Default (multiple host threads can use ::cudaSetDevice() with device simultaneously) >

deviceQuery, CUDA Driver = CUDART, CUDA Driver Version = 7.5, CUDA Runtime Version = 7.5, NumDevs = 1, Device0 = GeForce GTX 980
Result = PASS

6.安装atlas

安装命令：

sudo apt-get install libatlas-base-dev

7.安装opencv

(此处开始和参考博客有不同，之前按照原来博客的方法安装在编译的时候出了一些错误，也可能是版本问题)

我安装的版本是opencv3.0.0, 在opencv官网下载，解压。

装好opencv的各种依赖项

echo "--- Removing any pre-installed ffmpeg and x264"
sudo apt-get -qq remove ffmpeg x264 libx264-dev

echo "--- Installing dependency: "

sudo apt-get -y install libopencv-dev
sudo apt-get -y install build-essential
sudo apt-get -y install checkinstall
sudo apt-get -y install cmake
sudo apt-get -y install pkg-config
sudo apt-get -y install yasm
sudo apt-get -y install libtiff4-dev
sudo apt-get -y install libjpeg-dev
sudo apt-get -y install libjasper-dev
sudo apt-get -y install libavcodec-dev
sudo apt-get -y install libavformat-dev
sudo apt-get -y install libswscale-dev
sudo apt-get -y install libdc1394-22-dev
sudo apt-get -y install libxine-dev
sudo apt-get -y install libgstreamer0.10-dev
sudo apt-get -y install libgstreamer-plugins-base0.10-dev
sudo apt-get -y install libv4l-dev
sudo apt-get -y install python-dev
sudo apt-get -y install python-numpy
sudo apt-get -y install libtbb-dev
sudo apt-get -y install libqt4-dev
sudo apt-get -y install libgtk2.0-dev
sudo apt-get -y install libfaac-dev
sudo apt-get -y install libmp3lame-dev
sudo apt-get -y install libopencore-amrnb-dev
sudo apt-get -y install libopencore-amrwb-dev
sudo apt-get -y install libtheora-dev
sudo apt-get -y install libvorbis-dev
sudo apt-get -y install libxvidcore-dev
sudo apt-get -y install x264
sudo apt-get -y install v4l-utils
sudo apt-get -y install ffmpeg
sudo apt-get -y install unzip

（可以把代码复制下来，在opencv的文件夹里面创建一个dependencies.sh, 然后执行sudo bash dependencies.sh)

装好依赖项后，执行下面的语句安装：


mkdir build
cd build
cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local -D WITH_TBB=ON -D BUILD_NEW_PYTHON_SUPPORT=ON -D WITH_V4L=ON -D INSTALL_C_EXAMPLES=ON -D INSTALL_PYTHON_EXAMPLES=ON -D BUILD_EXAMPLES=ON -D WITH_OPENGL=ON -D WITH_CUFFT=ON -D WITH_CUDA=ON -D WITH_FFMPEG=OFF -D BUILD_TIFF=ON ..
make -j4
sudo make install
sudo sh -c 'echo "/usr/local/lib" > /etc/ld.so.conf.d/opencv.conf'
sudo ldconfig
echo "OpenCV 3.0.0-rc1 ready to be used"

cmake 里面的不同点是加入了-D BUILD_TIFF=ON ，这个如果没有加入的话在caffe的编译阶段会针对Opencv报错。

安装好Opencv可以找一个网上的例子测试一下是否安装成功。

8.安装python

到 anaconda官网下载，我装的是python2.7

切换到下载文件目录，执行：

sudo bash Anaconda2-4.1.1-Linux-x86_64.sh

添加路径：

执行sudo gedit /etc/ld.so.conf

添加内容：

/home/username/anaconda/lib 注意改username

执行sudo gedit ~/.bashrc

添加内容：

export LD_LIBRARY_PATH="/home/username/anaconda/lib:$LD_LIBRARY_PATH"

9.安装python依赖库

先下载个caffe包 github-caffe ,解压后进入caffe-master/python,执行：

for req in $(cat requirements.txt); do pip install $req; done

10.修改Makefile和Mafile.config

首先进入caffe-master目录，复制一份Makefile.config.examples

cp Makefile.config.example Makefile.config

改动1：打开Makefile,将第181行的：

LIBRARIES += glog gflags protobuf boost_system boost_filesystem m hdf5_hl hdf5

改为：

LIBRARIES += glog gflags protobuf boost_system boost_filesystem m hdf5_serial_hl hdf5_serial opencv_core opencv_highgui opencv_imgproc opencv_imgcodecs

这里的改动是为了避免出现hdf5.h错误。

改动2：打开Makefile.config，将第90行的：

INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include

改为：

INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial/

改动理由同上。

Makefile.config里面的设置也要根据自己的配置和装的版本，选择每一项是否去掉# 或加上#。

例如我用的是ubuntu15.04, python2.7, cuda7.5,opencv 3.0.0,因此我的Makefile.config为：

## Refer to http://caffe.berkeleyvision.org/installation.html
# Contributions simplifying and improving our build system are welcome!

# cuDNN acceleration switch (uncomment to build with cuDNN).
USE_CUDNN := 1

# CPU-only switch (uncomment to build without GPU support).
# CPU_ONLY := 1

# uncomment to disable IO dependencies and corresponding data layers
# USE_OPENCV := 0
# USE_LEVELDB := 0
# USE_LMDB := 0

# uncomment to allow MDB_NOLOCK when reading LMDB files (only if necessary)
#	You should not set this flag if you will be reading LMDBs with any
#	possibility of simultaneous read and write
# ALLOW_LMDB_NOLOCK := 1

# Uncomment if you're using OpenCV 3
 OPENCV_VERSION := 3

# To customize your choice of compiler, uncomment and set the following.
# N.B. the default for Linux is g++ and the default for OSX is clang++
# CUSTOM_CXX := g++

# CUDA directory contains bin/ and lib/ directories that we need.
CUDA_DIR := /usr/local/cuda
# On Ubuntu 14.04, if cuda tools are installed via
# "sudo apt-get install nvidia-cuda-toolkit" then use this instead:
# CUDA_DIR := /usr

# CUDA architecture setting: going with all of them.
# For CUDA < 6.0, comment the *_50 lines for compatibility.
CUDA_ARCH := -gencode arch=compute_20,code=sm_20 \
		-gencode arch=compute_20,code=sm_21 \
		-gencode arch=compute_30,code=sm_30 \
		-gencode arch=compute_35,code=sm_35 \
		-gencode arch=compute_50,code=sm_50 \
		-gencode arch=compute_50,code=compute_50

# BLAS choice:
# atlas for ATLAS (default)
# mkl for MKL
# open for OpenBlas
BLAS := atlas
# Custom (MKL/ATLAS/OpenBLAS) include and lib directories.
# Leave commented to accept the defaults for your choice of BLAS
# (which should work)!
# BLAS_INCLUDE := /path/to/your/blas
# BLAS_LIB := /path/to/your/blas

# Homebrew puts openblas in a directory that is not on the standard search path
# BLAS_INCLUDE := $(shell brew --prefix openblas)/include
# BLAS_LIB := $(shell brew --prefix openblas)/lib

# This is required only if you will compile the matlab interface.
# MATLAB directory should contain the mex binary in /bin.
# MATLAB_DIR := /usr/local
# MATLAB_DIR := /Applications/MATLAB_R2012b.app

# NOTE: this is required only if you will compile the python interface.
# We need to be able to find Python.h and numpy/arrayobject.h.
#PYTHON_INCLUDE := /usr/include/python2.7 \
		/usr/lib/python2.7/dist-packages/numpy/core/include
# Anaconda Python distribution is quite popular. Include path:
# Verify anaconda location, sometimes it's in root.
 ANACONDA_HOME := $(HOME)/anaconda
 PYTHON_INCLUDE := $(ANACONDA_HOME)/include \
		 $(ANACONDA_HOME)/include/python2.7 \
		 $(ANACONDA_HOME)/lib/python2.7/site-packages/numpy/core/include \

# Uncomment to use Python 3 (default is Python 2)
# PYTHON_LIBRARIES := boost_python3 python3.5m
# PYTHON_INCLUDE := $(ANACONDA_HOME)/include/python3.5m \
      #           $(ANACONDA_HOME)/lib/python3.5/dist-packages/numpy/core/include

# We need to be able to find libpythonX.X.so or .dylib.
#PYTHON_LIB := /usr/lib
 PYTHON_LIB := $(ANACONDA_HOME)/lib

# Homebrew installs numpy in a non standard path (keg only)
# PYTHON_INCLUDE += $(dir $(shell python -c 'import numpy.core; print(numpy.core.__file__)'))/include
# PYTHON_LIB += $(shell brew --prefix numpy)/lib

# Uncomment to support layers written in Python (will link against Python libs)
# WITH_PYTHON_LAYER := 1

# Whatever else you find you need goes here.
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial/
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib

# If Homebrew is installed at a non standard location (for example your home directory) and you use it for general dependencies
# INCLUDE_DIRS += $(shell brew --prefix)/include
# LIBRARY_DIRS += $(shell brew --prefix)/lib

# Uncomment to use `pkg-config` to specify OpenCV library paths.
# (Usually not necessary -- OpenCV libraries are normally installed in one of the above $LIBRARY_DIRS.)
# USE_PKG_CONFIG := 1

# N.B. both build and distribute dirs are cleared on `make clean`
BUILD_DIR := build
DISTRIBUTE_DIR := distribute

# Uncomment for debugging. Does not work on OSX due to https://github.com/BVLC/caffe/issues/171
# DEBUG := 1

# The ID of the GPU that 'make runtest' will use to run unit tests.
TEST_GPUID := 0

# enable pretty build (comment to see full commands)
Q ?= @

里面的内容一定要检查清楚，尤其检查phthon的目录，lib的路径有没有错误，这些有可能导致接下来编译可以通过但是安装python接口的时候出现找不到python.h的错误等等。

11.编译caffe

所有都整理完成之后就进入编译：

make all -j4  
make test  
make runtest

如果编译中出现什么错误，需要重新编译，一定要先make clean。

12.编译python wrapper

虽然这里命令很简单：

sudo make pycaffe

但是编译完成后，进入ipython, 输入import caffe的时候会发现出现了

ImportError: No module named google.protobuf

这是因为protobuf版本过低的原因，需要手动装一下protobuf：

先到 github-protobuf 下载，解压。

执行：

./autogen.sh
./configure
make
make check
sudo make install

发现./autogen.sh的时候可能会有错误，这是因为curl没装好，可以选择修复，也可以选择手动下载autogen.sh里面提到的两个包，手动解压，然后将他们按照autogen.sh里面要求重命名和移动。将gmock-release的那个包重命名为gmock，将gtest-release的包重命名为gtest，放到gmock里面。再执行下面的命令

编译完成后，去到python的目录，执行

sudo python setup.py install

这样，整个接口就完成了，再重新打开ipython, import caffe就不会再报错。

至此，整个caffe的安装配置工作完成！

huangjx07

关注

0
点赞
踩
1

收藏

觉得还不错? 一键收藏
0
评论
caffe的搭建过程以及遇上的各种问题的汇总

Caffe的搭建过程以及遇上的各种问题的汇总：ubuntu15.04+CUDA7.5+opencv3.0.0+python2.7整个搭建过程参考 http://blog.csdn.net/ubunfans/article/details/47724341，过程中遇上了各种花式的问题，废了两天终于将caffe框架搭好。整理一下过程和遇到的问题，给自己长长经验。
复制链接

扫一扫