Prerequisites
Common packages
- CUDA 8.0 & cuDNN 5.1
- ATLAS or OpenBLAS
- Boost
- protobuf
- glog
- gflags
- hdf5
- OpenCV 2.4.x
- lmdb
- leveldb
以上packages已由管理员通过apt-get安装,同学可根据需要在自己的.bashrc中添加环境变量,如:
export LD_LIBRARY_PATH=/usr/local/cuda/lib64:$LD_LIBRARY_PATH
-
注意
- GPU73默认使用GCC-4.7.3,使用GCC-4.9.x及以上版本的同学,如果遇到CXXABI版本不兼容导致链接出错,请自行编译安装protobuf、glog、gflags、lmdb、leveldb。
Python 2.7.x
使用PyCaffe的同学请在自己的$HOME
下安装Miniconda2,并将YOUR_MINICONDA_PATH/bin
加入到PATH
环境变量中,使用conda管理自己的python packages。
使用校内的conda源:
conda config --add channels 'https://mirrors4.tuna.tsinghua.edu.cn/anaconda/pkgs/free/'
conda config --remove channels defaults
conda config --set show_channel_urls yes
安装所需的python packages,如numpy、matplotlib、scikit-image、pyyaml、protobuf、easydict、opencv等。
-
注意
-
- easydict需要通过verydeep channel安装,即:
conda install -c verydeep easydict
- protobuf、opencv会和系统中的protobuf、opencv产生冲突,安装了这两个packages的同学请参考下面的步骤编译安装自己的protobuf和opencv。
- 编译OpenCV时如果编译了python2 lib,只需要将cv2.so添加到
PYTHONPATH
中,无需安装opencv package。
Protobuf
未使用GCC > 4.9 && 未安装protobuf python package的同学无需编译安装protobuf,可跳过本节。
下载并解压protobuf:
wget https://github.com/google/protobuf/archive/v3.1.0.tar.gz
tar -xzf v3.1.0.tar.gz
cd protobuf-3.1.0
生成配置文件(该过程需要automake、autoconf、libtool等工具,管理员已安装)并编译安装:
./autogen.sh
./configure --prefix=YOUR_PROTOBUF_DIRECTORY
make -j
make install
添加环境变量:
export PATH=YOUR_PROTOBUF_DIRECTORY/bin:$PATH
export LD_LIBRARY_PATH=YOUR_PROTOBUF_DIRECTORY/lib:$LD_LIBRARY_PATH
OpenCV 3.x
不需要编译OpenCV的同学可以跳过本节。
下载OpenCV并解压:
wget https://github.com/Itseez/opencv/archive/3.1.0.zip
unzip 3.1.0.zip
cd opencv-3.1.0
OpenCV使用CMake进行工程管理,借助CMake工具可以自动发现依赖项,并生成Makefile文件进行编译(在Windows下可生成Visual Studio的.sln)。然而CMake并不能自动发现安装在home下的miniconda,因此需要手工指定一些变量:
mkdir build
cd build
cmake -DCMAKE_BUILD_TYPE=RELEASE \
-DCMAKE_INSTALL_PREFIX=YOUR_OPENCV_PATH \
..
cmake -DPYTHON2_EXECUTABLE=$(which python) \
-DPYTHON2_INCLUDE_DIR=$(python -c "from distutils.sysconfig import get_python_inc; print(get_python_inc())") \
-DPYTHON2_LIBRARY=$(python -c 'import sys; from distutils import sysconfig; sys.stdout.write("/".join(map(sysconfig.get_config_var, ("LIBDIR", "INSTSONAME"))))') \
-DPYTHON2_PACKAGES_PATH=$(python -c "from distutils.sysconfig import get_python_lib; print(get_python_lib())") \
..
make -j
make install
如果在编译GraphCut时遇到错误,请参考Issue #6510解决。
手工安装cv2.so:
mkdir YOUR_OPENCV_PATH/python
cp ./lib/cv2.so YOUR_OPENCV_PATH/python/
chrpath -r '../lib:/usr/local/cuda/lib64:' YOUR_OPENCV_PATH/python/cv2.so
添加环境变量:
export LD_LIBRARY_PATH=YOUR_OPENCV_PATH/lib:$LD_LIBRARY_PATH
export PYTHONPATH=YOUR_OPENCV_PATH/python:$PYTHONPATH
Caffe & PyCaffe
Compile Caffe & PyCaffe
获取最新版本:
git clone https://github.com/BVLC/caffe.git
或者:
wget https://github.com/BVLC/caffe/archive/master.zip
unzip caffe-master.zip
参考Makefile.config.example示例文件编写自己的Makefile.config,注意修改以下地方:
- 取消注释:USE_CUDNN := 1
- (建议)取消注释:OPENCV_VERSION := 3
- (建议,需自行编译OpenBLAS,方法和Protobuf相同)BLAS库:BLAS := open
- (按需)Python设置:
# NOTE: this is required only if you will compile the python interface.
# We need to be able to find Python.h and numpy/arrayobject.h.
#PYTHON_INCLUDE := /usr/include/python2.7 \
# /usr/lib/python2.7/dist-packages/numpy/core/include
# Anaconda Python distribution is quite popular. Include path:
# Verify anaconda location, sometimes it's in root.
ANACONDA_HOME := YOUR_MINICONDA_PATH
PYTHON_INCLUDE := $(ANACONDA_HOME)/include \
$(ANACONDA_HOME)/include/python2.7 \
$(ANACONDA_HOME)/lib/python2.7/site-packages/numpy/core/include \
# We need to be able to find libpythonX.X.so or .dylib.
#PYTHON_LIB := /usr/lib
PYTHON_LIB := $(ANACONDA_HOME)/lib
- (按需)Python Layer设置:WITH_PYTHON_LAYER := 1
- 其他非默认位置的依赖项需要手工添加到INCLUDE_DIRS和LIBRARY_DIRS中,如:
INCLUDE_DIRS := YOUR_PROTOBUF_PATH/include YOUR_OPENCV_PATH/include $(PYTHON_INCLUDE) /usr/local/include
LIBRARY_DIRS := YOUR_PROTOBUF_PATH/lib YOUR_OPENCV_PATH/include $(PYTHON_LIB) /usr/local/lib /usr/lib
保存后,即可编译:
make -j
make pycaffe
如果遇到caffe.pb.h无法找到,请手工使用protoc对caffe.proto进行转换:
protoc ./src/caffe/proto/caffe.proto --cpp_out=.
mkdir ./include/caffe/proto
mv ./src/caffe/proto/caffe.pb.h ./include/caffe/proto/
Check Shared Library Dependencies
执行:
ldd ./build/lib/libcaffe.so.1.0.0-rc3
如果有not found的依赖项,请尝试将其所在目录添加到LD_LIBRARY_PATH
环境变量中。
-
注意
-
不要将
YOUR_MINICONDA_PATH/lib
添加到LD_LIBRARY_PATH中!Miniconda自带的libstdc++.so等库将掩盖系统库,造成严重混乱。
Try It Out
依次执行:
./data/mnist/get_mnist.sh
./examples/mnist/create_mnist.sh
./examples/mnist/train_lenet.sh
-
注意
-
可以使用
nvidia-smi
查看GPU负载情况,修改./examples/mnist/train_lenet.sh
指定空闲的GPU进行训练,如:
./build/tools/caffe train --gpu=1 --solver=examples/mnist/lenet_solver.prototxt $@
MatCaffe
Sorry, Matlab is missing on GPU73…
我们将尽快更新。