本文记录编译安装Caffe及python虚拟环境的pycaffe接口.
文章目录
1. 环境配置
访问caffe.berkeleyvision.org或者github.com/BVLC/caffe.
点击installation查看Caffe官方安装教程.
1.1 CUDA环境
1.2 Python虚拟环境
系统已经安装python3.6和virtualenv/virtualenvwrapper.
建立虚拟环境.
$ mkvirtualenv --no-site-packages -p python3.6 caffe-master
虚拟环境名称是caffe-master，它的路径在~/.virtualenvs/caffe-master
$ cd ~/.virtualenvs/caffe-master
$ ls
bin include lib man share
2. 安装依赖项
2.1 安装所需库文件
sudo apt-get install libprotobuf-dev liblmdb-dev libleveldb-dev libsnappy-dev libgflags-dev libgoogle-glog-dev libhdf5-serial-dev protobuf-compiler -y
2.2 安装BLAS
ATLAS
$ sudo apt-get install libatlas-base-dev -y
2.3 安装 boost_python3
Ubuntu16.04的apt提供python2.7和python3.5的boost_python，使用boost 1.58.0编译. python3.6 的boost_python需要boost源码编译.
使用下面命令安装python2.7和python3.5的boost_python.
sudo apt-get install --no-install-recommends libboost-all-dev -y
默认安装路径在 /usr/lib/x86_64-linux-gnu 目录下 .
$ cd /usr/lib/x86_64-linux-gnu
$ ll | grep 'libboost_python'
lrwxrwxrwx 1 root root 22 Мог 14 2016 libboost_python.a -> libboost_python-py27.a
-rw-r--r-- 1 root root 673046 Мог 14 2016 libboost_python-py27.a
lrwxrwxrwx 1 root root 30 Мог 14 2016 libboost_python-py27.so -> libboost_python-py27.so.1.58.0
-rw-r--r-- 1 root root 310128 Мог 14 2016 libboost_python-py27.so.1.58.0
-rw-r--r-- 1 root root 664980 Мог 14 2016 libboost_python-py35.a
lrwxrwxrwx 1 root root 30 Мог 14 2016 libboost_python-py35.so -> libboost_python-py35.so.1.58.0
-rw-r--r-- 1 root root 306000 Мог 14 2016 libboost_python-py35.so.1.58.0
lrwxrwxrwx 1 root root 23 Мог 14 2016 libboost_python.so -> libboost_python-py27.so
源码编译安装python3.6的libboost_python3.
在boost.org下载 boost 1.58.0. 或者直接点击这里下载boost_1_58_0.tar.gz.
$ tar -xzvf boost_1_58_0.tar.gz
$ cd boost_1_58_0
设置编译选项. 只编译python库.
$ ./bootstrap.sh --with-libraries=python --with-toolset=gcc --with-python=/usr/bin/python3.6 --with-python-version=3.6 --with-python-root=/usr/local/lib/python3.6
编译. 指定python环境的库文件路径.
$ sudo ./b2 --with-python include="/usr/include/python3.6m"
编译得到的库文件在 .../boost_1_58_0/stage/lib/ 下.
$ ll ~/Downloads/boost_1_58_0/stage/lib/
total 2104
drwxr-xr-x 2 root root 4096 Бич 18 17:24 ./
drwxrwxr-x 3 clyde clyde 4096 Бич 18 17:24 ../
-rw-r--r-- 1 root root 670236 Бич 18 17:24 libboost_python3.a
lrwxrwxrwx 1 root root 26 Бич 18 17:24 libboost_python3.so -> libboost_python3.so.1.58.0*
-rwxr-xr-x 1 root root 397768 Бич 18 17:24 libboost_python3.so.1.58.0*
-rw-r--r-- 1 root root 670236 Бич 18 17:24 libboost_python.a
lrwxrwxrwx 1 root root 25 Бич 18 17:24 libboost_python.so -> libboost_python.so.1.58.0*
-rwxr-xr-x 1 root root 397768 Бич 18 17:24 libboost_python.so.1.58.0*
复制编译结果到/usr/lib/x86_64-linux-gnu目录下.
$ sudo cp ~/Downloads/boost_1_58_0/stage/lib/libboost_python3.* /usr/lib/x86_64-linux-gnu/
删除libboost_python-py36.so.1.58.0的可执行权限.
$ sudo chmod -x /usr/lib/x86_64-linux-gnu/libboost_python3.so*
$ ll /usr/lib/x86_64-linux-gnu/ | grep 'libboost_python'
-rw-r--r-- 1 root root 670236 Бич 18 18:09 libboost_python3.a
-rw-r--r-- 1 root root 397768 Бич 18 18:09 libboost_python3.so
-rw-r--r-- 1 root root 397768 Бич 18 18:09 libboost_python3.so.1.58.0
lrwxrwxrwx 1 root root 22 Мог 14 2016 libboost_python.a -> libboost_python-py27.a
-rw-r--r-- 1 root root 673046 Мог 14 2016 libboost_python-py27.a
lrwxrwxrwx 1 root root 30 Мог 14 2016 libboost_python-py27.so -> libboost_python-py27.so.1.58.0
-rw-r--r-- 1 root root 310128 Мог 14 2016 libboost_python-py27.so.1.58.0
-rw-r--r-- 1 root root 664980 Мог 14 2016 libboost_python-py35.a
lrwxrwxrwx 1 root root 30 Мог 14 2016 libboost_python-py35.so -> libboost_python-py35.so.1.58.0
-rw-r--r-- 1 root root 306000 Мог 14 2016 libboost_python-py35.so.1.58.0
lrwxrwxrwx 1 root root 23 Мог 14 2016 libboost_python.so -> libboost_python-py27.so
lrwxrwxrwx 1 root root 30 Бич 18 18:01 libboost_python.so.1.58.0 -> libboost_python-py27.so.1.58.0
2.4 安装 OpenCV3
使用OpenCV3需要手动编译安装.
如果仅使用3.0以下版本的OpenCV库，Ubuntu16.04可以通过apt安装libopencv-2.4.
$ sudo apt install libopencv-dev -y
点击installation查看官方安装教程.
安装编译组件.
$ sudo apt-get install build-essential
安装必需项.
$ sudo apt-get install cmake git libgtk2.0-dev pkg-config libavcodec-dev libavformat-dev libswscale-dev
安装可选项.
sudo apt-get install python-dev libtbb2 libtbb-dev libjpeg-dev libpng-dev libtiff-dev libjasper-dev libdc1394-22-dev
在opencv.org/releases/下载 OpenCV-3.4或者点击这里下载opencv-3.4.11.zip.
$ unzip opencv-3.4.11.zip
$ cd opencv-3.4.11.zip
$ mkdir build
$ cd build/
$ cmake -D CMAKE_BUILD_TYPE=RELEASE -D CMAKE_INSTALL_PREFIX=/usr/local ..
$ make -j6 # runs 6 jobs in parallel
$ sudo make install
查看安装路径.
$ pkg-config --cflags --libs opencv
-I/usr/local/include/opencv -I/usr/local/include -L/usr/local/lib -lopencv_ml -lopencv_shape -lopencv_superres -lopencv_stitching -lopencv_videostab -lopencv_video -lopencv_dnn -lopencv_objdetect -lopencv_calib3d -lopencv_photo -lopencv_features2d -lopencv_flann -lopencv_highgui -lopencv_videoio -lopencv_imgcodecs -lopencv_imgproc -lopencv_core
添加共享库设置
编辑/etc/ld.so.conf.d/opencv.conf文件.
$ sudo vim /etc/ld.so.conf.d/opencv.conf
添加下面内容.
/usr/local/lib
激活设置
$ sudo ldconfig
添加环境变量
编辑/etc/bash.bashrc文件.
$ sudo vim /etc/bash.bashrc
在末尾添加下面2行内容.
PKG_CONFIG_PATH=$PKG_CONFIG_PATH:/usr/local/lib/pkgconfig
export PKG_CONFIG_PATH
激活设置
$ source /etc/bash.bashrc
$ source ~/.bashrc
如果ldconfig命令可以搜索到opencv3.4说明设置已经生效.
$ ldconfig -v | grep opencv | grep 3.4
/sbin/ldconfig.real: Path `/usr/local/lib' given more than once
/sbin/ldconfig.real: Path `/usr/lib/nvidia-384' given more than once
/sbin/ldconfig.real: Path `/usr/lib32/nvidia-384' given more than once
/sbin/ldconfig.real: Path `/lib/x86_64-linux-gnu' given more than once
/sbin/ldconfig.real: Path `/usr/lib/x86_64-linux-gnu' given more than once
/sbin/ldconfig.real: /usr/local/cuda-9.0/targets/x86_64-linux/lib/libcudnn.so.7 is not a symbolic link
/sbin/ldconfig.real: /lib/x86_64-linux-gnu/ld-2.23.so is the dynamic linker, ignoring
/sbin/ldconfig.real: Cannot stat /usr/lib32/nvidia-384/libOpenGL.so: No such file or directory
/sbin/ldconfig.real: /lib32/ld-2.23.so is the dynamic linker, ignoring
/sbin/ldconfig.real: Cannot stat /usr/lib/libopen-pal.so: No such file or directory
/sbin/ldconfig.real: Cannot stat /usr/lib/libmca_common_sm.so: No such file or directory
/sbin/ldconfig.real: Cannot stat /usr/lib/libopen-rte.so: No such file or directory
/sbin/ldconfig.real: Cannot stat /usr/lib/liboshmem.so: No such file or directory
/sbin/ldconfig.real: Can't create temporary cache file /etc/ld.so.cache~: Permission denied
libopencv_photo.so.3.4 -> libopencv_photo.so.3.4.11
libopencv_features2d.so.3.4 -> libopencv_features2d.so.3.4.11
libopencv_imgcodecs.so.3.4 -> libopencv_imgcodecs.so.3.4.11
libopencv_stitching.so.3.4 -> libopencv_stitching.so.3.4.11
libopencv_flann.so.3.4 -> libopencv_flann.so.3.4.11
libopencv_ml.so.3.4 -> libopencv_ml.so.3.4.11
libopencv_videostab.so.3.4 -> libopencv_videostab.so.3.4.11
libopencv_superres.so.3.4 -> libopencv_superres.so.3.4.11
libopencv_imgproc.so.3.4 -> libopencv_imgproc.so.3.4.11
libopencv_video.so.3.4 -> libopencv_video.so.3.4.11
libopencv_objdetect.so.3.4 -> libopencv_objdetect.so.3.4.11
libopencv_videoio.so.3.4 -> libopencv_videoio.so.3.4.11
libopencv_highgui.so.3.4 -> libopencv_highgui.so.3.4.11
libopencv_dnn.so.3.4 -> libopencv_dnn.so.3.4.11
libopencv_shape.so.3.4 -> libopencv_shape.so.3.4.11
libopencv_calib3d.so.3.4 -> libopencv_calib3d.so.3.4.11
libopencv_core.so.3.4 -> libopencv_core.so.3.4.11
3. 编译 Caffe
3.1 下载Caffe源码
点击caffe-1.0下载源码包.
3.2 设置Makefile.config
$ zunip caffe-master.zip
$ cd caffe-master
$ cp Makefile.config.example Makefile.config
编辑Makefile.config.
$ vim Makefile.config
我使用的配置如下：
## Refer to http://caffe.berkeleyvision.org/installation.html
# Contributions simplifying and improving our build system are welcome!
# cuDNN acceleration switch (uncomment to build with cuDNN).
USE_CUDNN := 1
# CPU-only switch (uncomment to build without GPU support).
# CPU_ONLY := 1
# uncomment to disable IO dependencies and corresponding data layers
# USE_OPENCV := 1
# USE_LEVELDB := 0
# USE_LMDB := 0
# This code is taken from https://github.com/sh1r0/caffe-android-lib
# USE_HDF5 := 0
# uncomment to allow MDB_NOLOCK when reading LMDB files (only if necessary)
# You should not set this flag if you will be reading LMDBs with any
# possibility of simultaneous read and write
ALLOW_LMDB_NOLOCK := 1
# Uncomment if you're using OpenCV 3
OPENCV_VERSION := 3
# To customize your choice of compiler, uncomment and set the following.
# N.B. the default for Linux is g++ and the default for OSX is clang++
# CUSTOM_CXX := g++
# CUDA directory contains bin/ and lib/ directories that we need.
CUDA_DIR := /usr/local/cuda
# On Ubuntu 14.04, if cuda tools are installed via
# "sudo apt-get install nvidia-cuda-toolkit" then use this instead:
# CUDA_DIR := /usr
# CUDA architecture setting: going with all of them.
# For CUDA < 6.0, comment the *_50 through *_61 lines for compatibility.
# For CUDA < 8.0, comment the *_60 and *_61 lines for compatibility.
# For CUDA >= 9.0, comment the *_20 and *_21 lines for compatibility.
# CUDA_ARCH := -gencode arch=compute_20,code=sm_20 \
# -gencode arch=compute_20,code=sm_21
CUDA_ARCH := -gencode arch=compute_30,code=sm_30 \
-gencode arch=compute_35,code=sm_35 \
-gencode arch=compute_50,code=sm_50 \
-gencode arch=compute_52,code=sm_52 \
-gencode arch=compute_60,code=sm_60 \
-gencode arch=compute_61,code=sm_61 \
-gencode arch=compute_61,code=compute_61
# BLAS choice:
# atlas for ATLAS (default)
# mkl for MKL
# open for OpenBlas
BLAS := atlas
# Custom (MKL/ATLAS/OpenBLAS) include and lib directories.
# Leave commented to accept the defaults for your choice of BLAS
# (which should work)!
# BLAS_INCLUDE := /path/to/your/blas
# BLAS_LIB := /path/to/your/blas
# Homebrew puts openblas in a directory that is not on the standard search path
# BLAS_INCLUDE := $(shell brew --prefix openblas)/include
# BLAS_LIB := $(shell brew --prefix openblas)/lib
# This is required only if you will compile the matlab interface.
# MATLAB directory should contain the mex binary in /bin.
# MATLAB_DIR := /usr/local
# MATLAB_DIR := /Applications/MATLAB_R2012b.app
# NOTE: this is required only if you will compile the python interface.
# We need to be able to find Python.h and numpy/arrayobject.h.
#PYTHON_INCLUDE := /usr/include/python2.7 \
# /usr/lib/python2.7/dist-packages/numpy/core/include
# Anaconda Python distribution is quite popular. Include path:
# Verify anaconda location, sometimes it's in root.
ANACONDA_HOME := $(HOME)/.virtualenvs/caffe-master
PYTHON_INCLUDE := $(ANACONDA_HOME)/include \
$(ANACONDA_HOME)/include/python3.6m \
$(ANACONDA_HOME)/lib/python3.6/site-packages/numpy/core/include
# Uncomment to use Python 3 (default is Python 2)
PYTHON_LIBRARIES := boost_python3 python3.6m
# PYTHON_INCLUDE := /usr/include/python3.5m \
# /usr/lib/python3.5/dist-packages/numpy/core/include
# We need to be able to find libpythonX.X.so or .dylib.
PYTHON_LIB := /usr/lib
# PYTHON_LIB := $(ANACONDA_HOME)/lib
# Homebrew installs numpy in a non standard path (keg only)
# PYTHON_INCLUDE += $(dir $(shell python -c 'import numpy.core; print(numpy.core.__file__)'))/include
# PYTHON_LIB += $(shell brew --prefix numpy)/lib
# Uncomment to support layers written in Python (will link against Python libs)
WITH_PYTHON_LAYER := 1
# Whatever else you find you need goes here.
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib \
/usr/lib/x86_64-linux-gnu/hdf5/serial
# If Homebrew is installed at a non standard location (for example your home directory) and you use it for general dependencies
# INCLUDE_DIRS += $(shell brew --prefix)/include
# LIBRARY_DIRS += $(shell brew --prefix)/lib
# NCCL acceleration switch (uncomment to build with NCCL)
# https://github.com/NVIDIA/nccl (last tested version: v1.2.3-1+cuda8.0)
# USE_NCCL := 1
# Uncomment to use `pkg-config` to specify OpenCV library paths.
# (Usually not necessary -- OpenCV libraries are normally installed in one of the above $LIBRARY_DIRS.)
# USE_PKG_CONFIG := 1
# N.B. both build and distribute dirs are cleared on `make clean`
BUILD_DIR := build
DISTRIBUTE_DIR := distribute
# Uncomment for debugging. Does not work on OSX due to https://github.com/BVLC/caffe/issues/171
# DEBUG := 1
# The ID of the GPU that 'make runtest' will use to run unit tests.
TEST_GPUID := 0
# enable pretty build (comment to see full commands)
Q ?= @
3.3 配置pycaffe接口依赖
$ sudo apt-get install python3.6-dev -y
修改python虚拟环境启动项
$ vim ~/.virtualenvs/caffe-master/bin/activate
在末尾添加下面内容. {path_to}/caffe-master/python表示caffe-master/python目录的绝对路径.
export PYTHONPATH={path_to}/caffe-master/python:$PYTHONPATH
激活python虚拟环境
$ workon caffe-master
安装所需的python包
(caffe-master)$ pip install -r python/requirements.txt
3.4 编译caffe
编译
$ make all -j6
验证
$ make test -j6
$ make runtest -j6
验证通过
[----------] Global test environment tear-down
[==========] 2207 tests from 285 test cases ran. (320839 ms total)
[ PASSED ] 2207 tests.
caffe 编译成功.
3.5 编译pycaffe
编译
make pycaffe -j6
验证
$ sudo apt-get install graphviz
(caffe-master)$ pip install pydot graphviz
(caffe-master)$ make pytest -j6
验证通过
----------------------------------------------------------------------
Ran 52 tests in 8.779s
OK
pycaffe 编译成功.
2489
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
