搭建caffe环境及训练模型

Mac OS搭建caffe环境

 

一、相关软件安装

1、        安装CUDA8

（1）   安装xcode及命令行

（2）   安装CUDA的dmg包

（3）   配置环境变量并source ~/.bash_profile

export CUDA_ROOT=/Developer/NVIDIA/CUDA-8.0

export PATH=$CUDA_ROOT/bin${PATH:+:${PATH}}

export DYLD_LIBRARY_PATH=$CUDA_ROOT/lib${DYLD_LIBRARY_PATH:+:${DYLD_LIBRARY_PATH}}

export LD_LIBRARY_PATH=$CUDA_ROOT/lib:$LD_LIBRARY_PATH

 

（4）   执行nvcc –V验证安装

 

2、        安装必要的包

brew install –vd snappy leveldb gflags glog szip lmdb

brew tap homebrew/science

brew install hdf5 opencv

 

3、        brew edit opencv并修改两行

-DPYTHON_LIBRARY=#{py_prefix}/lib/libpython2.7.dylib

-DPYTHON_INCLUDE_DIR=#{py_prefix}/include/python2.7

 

4、        安装protobuf和boost-python

brew install --build-from-source --with-python –vd protobuf

brew install --build-from-source -vd boost boost-python

 

5、下载NVIDIA Mac drivers

 

二、caffe安装

1、下载caffe

git clone https://github.com/BVLC/caffe.git

 

2、打开Makefile.config, 配置BLAS

BLAS_INCLUDE := /Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.12.sdk/System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/Headers

BLAS_LIB := /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A

 

去掉CPU_ONLY的注释

 

去掉USE_LEVELDB := 0的注释

 

修改PYTHON_INCLUDE的目录为python2.7的目录

 

三、编译caffe

make all

make test

make runtest

make pycaffe

make pytest

make distribute

会有一些warning, 需要安装什么包就安装什么包

1)“dot” not found in path问题需要安装graphviz，不用pip用brew安装

2) make pytest出现了如下问题:

注释掉/python/caffe/proto/caffe_pb2.py 所有 “syntax = proto2 “

 

3）需要进入python文件夹执行python才能import caffe

4）安装scikit-image库

5）pip不好使，就pip install –U pip

 

 

Centos 7安装Caffe

 

一、安装CUDA

wget https://developer.nvidia.com/compute/cuda/8.0/Prod2/local_installers/cuda-repo-rhel7-8-0-local-ga2-8.0.61-1.x86_64-rpm

rpm -i cuda-repo-rhel7-8-0-local-ga2-8.0.61-1.x86_64-rpm

yum clean all

yum install cuda

 

设置CUDA环境变量：

export LD_LIBRARY_PATH=/usr/local/cuda-8.0/lib64:/usr/local/cuda-8.0/extras/CUPTI/lib64:$LD_LIBRARY_PATH（指定共享库，编译opencv、cudasample等）

export CUDA_HOME=/usr/local/cuda-8.0/（指定CUDA安装路径）

export PATH=/usr/local/cuda-8.0/bin:$PATH（在任意环境下调用cuda可执行文件）

 

更新运行库缓存：

ldconfig

 

检验CUDA是否安装成功：

nvcc --version

 

GPU支持：

       安装Nvidia驱动：

根据GPU型号从相应网站下载驱动，例如使用NVIDIA Tesla M60，从NVIDIA网站选择对应的型号和操作系统，CUDA Toolkit版本，下载驱动文件，如NVIDIA-Linux-x86_64-375.66.run，运行驱动文件，根据提示安装：

sh  NVIDIA-Linux-x86_64-375.66.run

通过NVIDIA命令工具nvidia-smi检验GPU情况:

nvidia-smi

 

安装cuDNN：

从NVIDIA网站下载cuDNN 安装包，根据GPU及CUDA版本选择对应cuDNN版本，下载cuDNN v5.1 for CUDA8.0,解压拷贝到CUDA安装目录

cp include/* /usr/local/cuda/include

cp lib64/* /usr/local/cuda/lib64

 

二、yum安装依赖包

yum install protobuf-devel  leveldb-devel  snappy-devel  opencv-devel  boost-devel  hdf5-devel  gflags-devel  glog-devel  lmdb-devel  atlas-devel

n  cd  /usr/lib64/atlas

n  sudo ln -sv libsatlas.so.3.10 libcblas.so

n  sudo ln -sv libsatlas.so.3.10 libatlas.so

（如果有需要，可安装Openblas或者MKL作为atlas的备用）

 

二、非yum手动安装依赖包

 

安装 gflags：

git clone https://github.com/gflags/gflags

cd gflags

mkdir build && cd build

export CXXFLAGS="-fPIC" && cmake .. -DCMAKE_INSTALL_PREFIX=/data2/shixi_wuhao3/local -DBUILD_SHARED_LIBS=ON

make VERBOSE=1

sudo make install

 

安装 glog：

git clone https://github.com/google/glog

cd glog

./configure

make

automake --add-missing

sudo make install

 

安装 lmdb：

git clone https://github.com/LMDB/lmdb

cd lmdb/libraries/liblmdb

make

sudo make install

 

安装 hdf5：

wget https://support.hdfgroup.org/ftp/HDF5/current18/src/hdf5-1.8.18.tar.gz

tar -xvf hdf5-1.8.18.tar.gz

cd hdf5-1.8.18

./configure --prefix=/usr/local

make

sudo make install

 

安装protobuf

tar -xvf protobuf

cd protobuf

./configure --prefix=/usr/local/protobuf

make

make check

make install

 

安装boost

wget http://sourceforge.net/projects/boost/files/boost/1.59.0/boost_1_59_0.tar.gz

tar -zxvf boost_1_59_0.tar.gz

cd boost_1_59_0

./bootstrap.sh --prefix=path/to/install

./b2 install

这会把boost安装到path/to/install，若不指定，默认将安装到/usr/local目录，但需要权限。

 

安装opencv

leveldb要和snappy一起安装

 

三、下载caffe

git clone https://github.com/BVLC/caffe

 

四、安装Python依赖包（注意python2和python3的）

for req in $(cat caffe/python/requirements.txt); do pip install $req; done

 

五、编辑Makefile.config 文件，根据情况修改配置：

BLAS := atlas

BLAS_INCLUDE := /usr/include/atlas

BLAS_LIB := /usr/lib64/atlas

PYTHON_INCLUDE := /usr/local/include/python2.7 \

                /usr/local/lib/python2.7/site-packages/numpy/core/include

PYTHON_LIB := /usr/lib64

 

USE_CUDNN := 1（使用cuDNN）

 

（去掉CPU_ONLY的注释——只用CPU）

（去掉USE_LEVELDB := 0的注释——使用LevelDB）

 

根据情况确定是否修改Makefile.cofig中的相关位置路径

## Refer to http://caffe.berkeleyvision.org/installation.html

# Contributions simplifying and improving our build system are welcome!

 

# cuDNN acceleration switch (uncomment to build with cuDNN).

# USE_CUDNN := 1

"CuDNN是NVIDIA专门针对Deep Learning框架设计的一套GPU计算加速库，用于实现高性能的并行计算,在有GPU并且安装CuDNN的情况下可以打开即将注释去掉。"

 

# CPU-only switch (uncomment to build without GPU support).

#CPU_ONLY := 1

"表示是否用GPU,如果只有CPU这里要打开"

 

# uncomment to disable IO dependencies and corresponding data layers

USE_OPENCV := 1

"因为要用到OpenCV库所以要打开，下面这两个选项表示是选择Caffe的数据管理第三方库，两者都不打开 Caffe默认用的是LMDB，这两者均是嵌入式数据库管理系统编程库。"

# USE_LEVELDB := 0

# USE_LMDB := 0

 

# uncomment to allow MDB_NOLOCK when reading LMDB files (only if necessary)

#   You should not set this flag if you will be reading LMDBs with any

#   possibility of simultaneous read and write

# ALLOW_LMDB_NOLOCK := 1

"当需要读取LMDB文件时可以取消注释，默认不打开。"

 

# Uncomment if you're using OpenCV 3

OPENCV_VERSION := 2.4.10 （我的2.4.5）

"用pkg-config --modversion opencv命令查看opencv版本"

 

# To customize your choice of compiler, uncomment and set the following.

# N.B. the default for Linux is g++ and the default for OSX is clang++

# CUSTOM_CXX := g++

"linux系统默认使用g++编译器，OSX则是clang++。"

 

# CUDA directory contains bin/ and lib/ directories that we need.

CUDA_DIR := /usr/local/cuda

"CUDA的安装目录"

# On Ubuntu 14.04, if cuda tools are installed via

# "sudo apt-get install nvidia-cuda-toolkit" then use this instead:

# CUDA_DIR := /usr

 

# CUDA architecture setting: going with all of them.

# For CUDA < 6.0, comment the *_50 lines for compatibility.

CUDA_ARCH := -gencode arch=compute_20,code=sm_20 \

        -gencode arch=compute_20,code=sm_21 \

        -gencode arch=compute_30,code=sm_30 \

        -gencode arch=compute_35,code=sm_35 \

        -gencode arch=compute_50,code=sm_50 \

        -gencode arch=compute_50,code=compute_50

"这些参数需要根据GPU的计算能力

(http://blog.csdn.net/jiajunlee/article/details/52067962)来进行设置，6.0以下的版本不支持×_50的计算能力。"

 

# BLAS choice:

# atlas for ATLAS (default)

# mkl for MKL

# open for OpenBlas

BLAS := open

"如果用的是ATLAS计算库则赋值atlas，MKL计算库则用mkl赋值，OpenBlas则赋值open。"

 

# Custom (MKL/ATLAS/OpenBLAS) include and lib directories.

# Leave commented to accept the defaults for your choice of BLAS

# (which should work)!

BLAS_INCLUDE := /usr/local/OpenBlas/include

BLAS_LIB := /usr/local/OpenBlas/lib

"blas库安装目录"

 

# Homebrew puts openblas in a directory that is not on the standard search path

# BLAS_INCLUDE := $(shell brew --prefix openblas)/include

# BLAS_LIB := $(shell brew --prefix openblas)/lib

"如果不是安装在标准路径则要指明"

 

# This is required only if you will compile the matlab interface.

# MATLAB directory should contain the mex binary in /bin.

# MATLAB_DIR := /usr/local

# MATLAB_DIR := /Applications/MATLAB_R2012b.app

"matlab安装库的目录"

 

# NOTE: this is required only if you will compile the python interface.

# We need to be able to find Python.h and numpy/arrayobject.h.

PYTHON_INCLUDE := /usr/include/python2.7 \

        /usr/lib/python2.7/dist-packages/numpy/core/include

"python安装目录"

# Anaconda Python distribution is quite popular. Include path:

# Verify anaconda location, sometimes it's in root.

# ANACONDA_HOME := $(HOME)/anaconda

# PYTHON_INCLUDE := $(ANACONDA_HOME)/include \

        # $(ANACONDA_HOME)/include/python2.7 \

        # $(ANACONDA_HOME)/lib/python2.7/site-packages/numpy/core/include \

 

# Uncomment to use Python 3 (default is Python 2)

# PYTHON_LIBRARIES := boost_python3 python3.5m

# PYTHON_INCLUDE := /usr/include/python3.5m \

#                 /usr/lib/python3.5/dist-packages/numpy/core/include

 

# We need to be able to find libpythonX.X.so or .dylib.

PYTHON_LIB := /usr/lib

"python库位置"

# PYTHON_LIB := $(ANACONDA_HOME)/lib

 

# Homebrew installs numpy in a non standard path (keg only)

# PYTHON_INCLUDE += $(dir $(shell python -c 'import numpy.core; print(numpy.core.__file__)'))/include

# PYTHON_LIB += $(shell brew --prefix numpy)/lib

 

# Uncomment to support layers written in Python (will link against Python libs)

WITH_PYTHON_LAYER := 1

 

# Whatever else you find you need goes here.

INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include

LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib

 

# If Homebrew is installed at a non standard location (for example your home directory) and you use it for general dependencies

# INCLUDE_DIRS += $(shell brew --prefix)/include

# LIBRARY_DIRS += $(shell brew --prefix)/lib

 

# Uncomment to use `pkg-config` to specify OpenCV library paths.

# (Usually not necessary -- OpenCV libraries are normally installed in one of the above $LIBRARY_DIRS.)

# USE_PKG_CONFIG := 1

 

# N.B. both build and distribute dirs are cleared on `make clean`

BUILD_DIR := build

DISTRIBUTE_DIR := distribute

 

# Uncomment for debugging. Does not work on OSX due to https://github.com/BVLC/caffe/issues/171

# DEBUG := 1

 

# The ID of the GPU that 'make runtest' will use to run unit tests.

TEST_GPUID := 0

"所用的GPU的ID编号"

 

# enable pretty build (comment to see full commands)

Q ?= @

 

 

六、编译及测试：

make all

make test

make runtest

测试成功，结果如下：

 

 

编译caffe的python环境及测试：

make pycaffe

make pytest

这样进入python交互行import caffe成功即可

 

 

http://caffe.berkeleyvision.org/installation.html

 

pip安装future库

安装最新版的cudnn（美团有链接，可能要注册）

cmake要设置CPU参数OFF, cmake -DUSE_CUDA=OFF ..

git要加参数recursive

make install时要加sudo

在site-package目录下编写.pth来使得caffe在任何目录下都可以import

转载于:https://www.cnblogs.com/yinghuali/p/7497535.html