Python2.7与Python3.5共存Caffe

最新推荐文章于 2022-06-15 16:20:32 发布
最新推荐文章于 2022-06-15 16:20:32 发布
阅读量1.3k 收藏 3
点赞数 2
分类专栏： Caffe 文章标签： Caffe
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/zym19941119/article/details/88917405
版权
本文介绍了如何在Ubuntu16.04系统中同时配置Python2.7的NVCaffe和Python3.5的BVLC Caffe，包括修改Makefile.config，解决protobuf动态库冲突问题，以及确保每个Python版本能正确导入对应的Caffe库。详细步骤包括注释Python Include，静态编译protobuf等。
摘要由CSDN通过智能技术生成

先简单说一下我的环境：

我用的是Ubuntu16.04，电脑中有两个版本的Caffe，分别是NVCaffe-0.17-dev与BVLC Caffe 1.0版本。由于工作需要，两个版本的caffe都要用，因此想要尝试在Python2.7中使用NVCaffe，在Python3.5中使用BVLC Caffe。电脑装有CUDA9.0+CUDNNv7，显卡为GTX1080。经过一番折腾终于搞定，记录下来以备不时之需。

Python2.7下安装NVCaffe

NVCaffe的安装与正常安装一样，默认安装在Python2.7中，只需要按照自己的需要更改下Makefile.config即可，我的Makefile.config如下：

## Refer to http://caffe.berkeleyvision.org/installation.html
# Contributions simplifying and improving our build system are welcome!

# cuDNN acceleration switch (uncomment to build with cuDNN).
# cuDNN version 6 or higher is required.
USE_CUDNN := 1

# NCCL acceleration switch (uncomment to build with NCCL)
# See https://github.com/NVIDIA/nccl
# USE_NCCL := 1

# Builds tests with 16 bit float support in addition to 32 and 64 bit.
#TEST_FP16 := 1

# uncomment to disable IO dependencies and corresponding data layers
# USE_OPENCV := 0
# USE_LEVELDB := 0
# USE_LMDB := 0

# Uncomment if you're using OpenCV 3
#OPENCV_VERSION := 3

# To customize your choice of compiler, uncomment and set the following.
# N.B. the default for Linux is g++ and the default for OSX is clang++
# CUSTOM_CXX := g++

# CUDA directory contains bin/ and lib/ directories that we need.
CUDA_DIR := /usr/local/cuda
# On Ubuntu 14.04, if cuda tools are installed via
# "sudo apt-get install nvidia-cuda-toolkit" then use this instead:
# CUDA_DIR := /usr

# CUDA architecture setting: going with all of them.
CUDA_ARCH := 	\
		-gencode arch=compute_61,code=sm_61 \
		-gencode arch=compute_61,code=compute_61

# BLAS choice:
# atlas for ATLAS
# mkl for MKL
# open for OpenBlas - default, see https://github.com/xianyi/OpenBLAS
BLAS := open
# Custom (MKL/ATLAS/OpenBLAS) include and lib directories.
#BLAS_INCLUDE := /opt/OpenBLAS/include/
#BLAS_LIB := /opt/OpenBLAS/lib/

# Homebrew puts openblas in a directory that is not on the standard search path
# BLAS_INCLUDE := $(shell brew --prefix openblas)/include
# BLAS_LIB := $(shell brew --prefix openblas)/lib

# This is required only if you will compile the matlab interface.
# MATLAB directory should contain the mex binary in /bin.
# MATLAB_DIR := /usr/local
# MATLAB_DIR := /Applications/MATLAB_R2012b.app

# NOTE: this is required only if you will compile the python interface.
# We need to be able to find Python.h and numpy/arrayobject.h.
PYTHON_INCLUDE := /usr/include/python2.7 \
		/usr/local/lib/python2.7/dist-packages/numpy/core/include
# Anaconda Python distribution is quite popular. Include path:
# Verify anaconda location, sometimes it's in root.
# ANACONDA_HOME := $(HOME)/anaconda
# PYTHON_INCLUDE := $(ANACONDA_HOME)/include \
		# $(ANACONDA_HOME)/include/python2.7 \
		# $(ANACONDA_HOME)/lib/python2.7/site-packages/numpy/core/include \

# Uncomment to use Python 3 (default is Python 2)
#PYTHON_LIBRARIES := boost_python3 python3.5m
#PYTHON_INCLUDE := /usr/include/python3.5m \
#                 /usr/lib/python3.5/dist-packages/numpy/core/include

# We need to be able to find libpythonX.X.so or .dylib.
PYTHON_LIB := /usr/lib
# PYTHON_LIB := $(ANACONDA_HOME)/lib

# Homebrew installs numpy in a non standard path (keg only)
# PYTHON_INCLUDE += $(dir $(shell python -c 'import numpy.core; print(numpy.core.__file__)'))/include
# PYTHON_LIB += $(shell brew --prefix numpy)/lib

# Uncomment to support layers written in Python (will link against Python libs)
WITH_PYTHON_LAYER := 1

# Whatever else you find you need goes here.
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/include /usr/local/include /usr/include/hdf5/serial
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib /usr/lib/x86_64-linux-gnu/hdf5/serial /usr/lib/x86_64-linux-gnu

# If Homebrew is installed at a non standard location (for example your home directory) and you use it for general dependencies
# INCLUDE_DIRS += $(shell brew --prefix)/include
# LIBRARY_DIRS += $(shell brew --prefix)/lib

# Uncomment to use `pkg-config` to specify OpenCV library paths.
# (Usually not necessary -- OpenCV libraries are normally installed in one of the above $LIBRARY_DIRS.)
# USE_PKG_CONFIG := 1

BUILD_DIR := build
DISTRIBUTE_DIR := distribute

# Uncomment for debugging. Does not work on OSX due to https://github.com/BVLC/caffe/issues/171
# DEBUG := 1

# The ID of the GPU that 'make runtest' will use to run unit tests.
TEST_GPUID := 0

# enable pretty build (comment to see full commands)
Q ?= @

# shared object suffix name to differentiate branches
LIBRARY_NAME_SUFFIX := -nv

安装只需要执行以下命令即可：

make clean
make all -j12
make pycaffe

安装好后，进入python环境即可以成功import caffe。

zym@zym-uestc:~$ python
Python 2.7.12 (default, Nov 12 2018, 14:36:49) 
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import caffe
>>>

Python3.5下安装Caffe

同样需要更改Makefile.config，值得注意的是需要把默认的PYTHON_INCLUDE给注释掉，把Python3环境的PYTHON_INCLUDE和PYTHON_LIBRARIES取消注释，完整的Makefile.config如下：

## Refer to http://caffe.berkeleyvision.org/installation.html
# Contributions simplifying and improving our build system are welcome!

# cuDNN acceleration switch (uncomment to build with cuDNN).
USE_CUDNN := 1

# CPU-only switch (uncomment to build without GPU support).
# CPU_ONLY := 1

# uncomment to disable IO dependencies and corresponding data layers
# USE_OPENCV := 0
# USE_LEVELDB := 0
# USE_LMDB := 0
# This code is taken from https://github.com/sh1r0/caffe-android-lib
# USE_HDF5 := 0

# uncomment to allow MDB_NOLOCK when reading LMDB files (only if necessary)
#	You should not set this flag if you will be reading LMDBs with any
#	possibility of simultaneous read and write
# ALLOW_LMDB_NOLOCK := 1

# Uncomment if you're using OpenCV 3
# OPENCV_VERSION := 3

# To customize your choice of compiler, uncomment and set the following.
# N.B. the default for Linux is g++ and the default for OSX is clang++
# CUSTOM_CXX := g++

# CUDA directory contains bin/ and lib/ directories that we need.
CUDA_DIR := /usr/local/cuda
# On Ubuntu 14.04, if cuda tools are installed via
# "sudo apt-get install nvidia-cuda-toolkit" then use this instead:
# CUDA_DIR := /usr

# CUDA architecture setting: going with all of them.
# For CUDA < 6.0, comment the *_50 through *_61 lines for compatibility.
# For CUDA < 8.0, comment the *_60 and *_61 lines for compatibility.
# For CUDA >= 9.0, comment the *_20 and *_21 lines for compatibility.
CUDA_ARCH := -gencode arch=compute_30,code=sm_30 \
		-gencode arch=compute_35,code=sm_35 \
		-gencode arch=compute_50,code=sm_50 \
		-gencode arch=compute_52,code=sm_52 \
		-gencode arch=compute_60,code=sm_60 \
		-gencode arch=compute_61,code=sm_61 \
		-gencode arch=compute_61,code=compute_61

# BLAS choice:
# atlas for ATLAS (default)
# mkl for MKL
# open for OpenBlas
BLAS := atlas
# Custom (MKL/ATLAS/OpenBLAS) include and lib directories.
# Leave commented to accept the defaults for your choice of BLAS
# (which should work)!
# BLAS_INCLUDE := /path/to/your/blas
# BLAS_LIB := /path/to/your/blas

# Homebrew puts openblas in a directory that is not on the standard search path
# BLAS_INCLUDE := $(shell brew --prefix openblas)/include
# BLAS_LIB := $(shell brew --prefix openblas)/lib

# This is required only if you will compile the matlab interface.
# MATLAB directory should contain the mex binary in /bin.
# MATLAB_DIR := /usr/local
# MATLAB_DIR := /Applications/MATLAB_R2012b.app

# NOTE: this is required only if you will compile the python interface.
# We need to be able to find Python.h and numpy/arrayobject.h.
# PYTHON_INCLUDE := /usr/include/python2.7 \
#		/usr/lib/python2.7/dist-packages/numpy/core/include
# Anaconda Python distribution is quite popular. Include path:
# Verify anaconda location, sometimes it's in root.
# ANACONDA_HOME := $(HOME)/anaconda
# PYTHON_INCLUDE := $(ANACONDA_HOME)/include \
		# $(ANACONDA_HOME)/include/python2.7 \
		# $(ANACONDA_HOME)/lib/python2.7/site-packages/numpy/core/include

# Uncomment to use Python 3 (default is Python 2)
PYTHON_LIBRARIES := boost_python-py35 python3.5m
PYTHON_INCLUDE := /usr/include/python3.5m \
                /usr/lib/python3.5/dist-packages/numpy/core/include

# We need to be able to find libpythonX.X.so or .dylib.
PYTHON_LIB := /usr/lib
# PYTHON_LIB := $(ANACONDA_HOME)/lib

# Homebrew installs numpy in a non standard path (keg only)
# PYTHON_INCLUDE += $(dir $(shell python -c 'import numpy.core; print(numpy.core.__file__)'))/include
# PYTHON_LIB += $(shell brew --prefix numpy)/lib

# Uncomment to support layers written in Python (will link against Python libs)
WITH_PYTHON_LAYER := 1

# Whatever else you find you need goes here.
INCLUDE_DIRS := $(PYTHON_INCLUDE) /usr/local/include /usr/include/hdf5/serial/
LIBRARY_DIRS := $(PYTHON_LIB) /usr/local/lib /usr/lib /usr/lib/x86_64-linux-gnu /usr/lib/x86_64-linux-gnu/hdf5/serial       

# If Homebrew is installed at a non standard location (for example your home directory) and you use it for general dependencies
# INCLUDE_DIRS += $(shell brew --prefix)/include
# LIBRARY_DIRS += $(shell brew --prefix)/lib

# NCCL acceleration switch (uncomment to build with NCCL)
# https://github.com/NVIDIA/nccl (last tested version: v1.2.3-1+cuda8.0)
# USE_NCCL := 1

# Uncomment to use `pkg-config` to specify OpenCV library paths.
# (Usually not necessary -- OpenCV libraries are normally installed in one of the above $LIBRARY_DIRS.)
# USE_PKG_CONFIG := 1

# N.B. both build and distribute dirs are cleared on `make clean`
BUILD_DIR := build
DISTRIBUTE_DIR := distribute

# Uncomment for debugging. Does not work on OSX due to https://github.com/BVLC/caffe/issues/171
# DEBUG := 1

# The ID of the GPU that 'make runtest' will use to run unit tests.
TEST_GPUID := 0

# enable pretty build (comment to see full commands)
Q ?= @

同样

make clean
make all -j12
make pycaffe

当你以为你安装成功的时候，在Python3.5下使用Caffe时，会报错：

[libprotobuf ERROR google/protobuf/descriptor_database.cc:57] File already exists in database: caffe.proto
[libprotobuf FATAL google/protobuf/descriptor.cc:1018] CHECK failed: generated_database_->Add(encoded_file_descriptor, size):

这个问题是两个Caffe都使用了动态库的原因，因此需要将protobuf改为静态库。

具体的方法如下：

1、 在Caffe的Makefile文件中找到这一行：

LIBRARIES += glog gflags protobuf boost_system boost_filesystem m hdf5_hl hdf5

并删除行内的protobuf

2、 在Makefile中找到如下行：

LDFLAGS += $(foreach librarydir,$(LIBRARY_DIRS),-L$(librarydir)) $(PKG_CONFIG) \
                  $(foreach library,$(LIBRARIES),-l$(library))

下加入

LDFLAGS += -Wl,-Bstatic -lprotobuf -Wl,-Bdynamic

3、 卸载已安装的protobuf

sudo apt-get remove libprotobuf-dev protobuf-compiler
pip uninstall protobuf
pip3 uninstall protobuf

4、 重新安装protobuf

我使用的是protobuf2.6.1，下载地址为：
protobuf2.6.1

解压之后打开configure文件，并修改以下内容

if test "x${ac_cv_env_CFLAGS_set}" = "x"; then :
  CFLAGS=""
fi
if test "x${ac_cv_env_CXXFLAGS_set}" = "x"; then :
  CXXFLAGS=""
fi

为

if test "x${ac_cv_env_CFLAGS_set}" = "x"; then :
  CFLAGS="-fPIC"
fi
if test "x${ac_cv_env_CXXFLAGS_set}" = "x"; then :
  CXXFLAGS="-fPIC"
fi

随后就可以开始编译google protobuff，只不过在configure 时加上选项 configrue --disable-shared 编译成静态库。
步骤如下：

./configrue --disable-shared
make
make check
sudo make install
sudo ldconfig

由于之前已经使用过protobuf，也不需要再配环境变量什么的，在shell内可以看到

zym@zym-uestc:~$ protoc --version
libprotoc 2.6.1

即为安装成功。

随后安装python内的protobuf

pip install protobuf
pip3 install protobuf

现在，在Python2.7内可以使用caffe：

zym@zym-uestc:~$ python
Python 2.7.12 (default, Nov 12 2018, 14:36:49) 
[GCC 5.4.0 20160609] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import caffe
>>>

但是在Python3.5内需要先将BVLC Caffe的python path加入环境变量

zym@zym-uestc:~$ python3
Python 3.5.2 (default, Nov 12 2018, 13:43:14) 
[GCC 5.4.0 20160609] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import sys
>>> sys.path.insert(0, '/home/zym/caffe/python')
>>> import caffe
>>>

但至少可以使用了～

参考资料

https://blog.csdn.net/CareyTian/article/details/80353314
https://blog.csdn.net/derkampf/article/details/69660050
https://blog.csdn.net/u012516318/article/details/79435181

心态已炸_沉迷学习
关注
  • 2
    点赞
  • 踩
  • 3
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
专栏目录
win10在Anaconda3下面安装caffe框架（CPU）Python2.7从入门到放弃
朕不知如何疼你
08-24 1455
安装caffe 从入门到放弃 安装环境 Windows10专业版 Anaconda3，conda 4.7.12 Python 2.7 参考：https://blog.csdn.net/qq_35756383/article/details/102491852 博客很长，内容很详细，虽然是GPU版本但是参考价值很大。使用git从github上面下载代码，联想到cmd操作环境，决定试一试anaconda环境下再装一次。 环境名称：caffe 修改MSNC_VERSION和CMAK.
python2.7环境下安装caffe、pytorch(cpuonly)
momentum30的博客
06-04 1683
以下步骤在ubuntu16.04、18.04中测试通过 Caffe对Ubuntu系统中python2.7支持是最好的，曾尝试在anaconda中 创建python2.7和Python3.7环境的方式安装caffe，最终都以失败告终。以下文档记录了在ubuntu16.04/18.04系统python2.7环境下安装caffe、pytorch(cpu only)的过程。 如下安装过程中： sudo apt源我使用的源是 http://mirrors.aliyun.com/ubuntu pip安装过程中，
Ubuntu16.04下安装Caffe+python2.7教程
05-23
本教程为Ubuntu16.04下基于CPU的caffe的安装，python版本为2.7，经测试，可以使用。
windows +caffe+python2.7或者python3.5编译 。
燕儿归的专栏
02-15 1056
caffe 只支持python2.7和3.5。想要编译caffe需要vs2013和vs2015，并且cpu版和gpu版不通用。很坑。 python2.7和python3.5中的print方法 一个是 print "hello",一个是print("hello")。有时候需要报错咱们手动改。 前期编译的大部分No module错误都可以通过pip install 解决。（另外No module的时候最好到百度搜一下，有个别情况pip安装的包名和报错的包名不一样，但是他们俩是一个。） 我安装python3
win10-caffe-python2.7配置方法（简洁易懂！！）
Site1997的博客
01-14 6008
引言    Caffe，全称Convolutional Architecture for Fast Feature Embedding。是一种常用的深度学习框架，在视频、图像处理方面应用较多。借用《21天实战caffe》里面的一段很形象的描述：Caffe的万丈高楼（Net）是按照我们的设计图纸（prototxt），用Blob这些砖块筑成一层层（Layer）楼房，最后通过SGD方法（Solver）
caffe-window-python2.7编译包
02-18
caffe-window-python2.7编译包，刚编译完，里面有完整的编译需要的文件，不需要在下载一堆了，所以压缩包会比较大。下载后，找到windows文件夹下的Caffe.sln。强烈建议用vs2013打开，别的版本打开语法不兼容。配置项...
python2.7可以直接使用的caffer依赖包
12-12
Python 2.7 版本下的 Caffe 是一个流行的深度学习框架，由 Berkeley Vision and Learning Center（BVLC）开发。Caffe 提供了一个高效的、模块化的、可扩展的平台，用于构建、训练和部署卷积神经网络（CNNs）和其他...
Caffe:Visual Studio 2013 Release, CPU only, Python 2.7
03-27
与Python 2.7的兼容性意味着这个版本的Caffe适用于那些仍依赖于2.7版本的项目。 4. **Caffe的功能**：Caffe以其高效和快速的前向传播而闻名，特别是在图像分类和物体检测等任务上。其核心特性包括： - **模型定义*...
caffe python3.5
10-04
Caffe 是一个流行的深度学习框架，由伯克利视觉与学习中心（BVLC）开发，主要用 C++ 编写，但同时也提供了 Python 接口。在这个“caffe python3.5”的主题中，我们将深入探讨如何在 Python 3.5 的环境中安装和使用 ...
caffe编译python3.5需要的boost库
01-03
1. **Boost与Python 3.5集成**：在编译Caffe以支持Python 3.5时，首先需要确保你已经安装了Python 3.5以及其开发头文件（通常包含在`python3.5-dev`或类似包中）。接下来，你需要下载并编译Boost库，确保在编译时...
caffe_python依赖版本要求
11-10
caffe_python依赖版本要求,2018年，有些包升级太厉害，不支持2了。
win10+python2.7+caffe+vs2015
qiangzi的博客
10-17 973
1、下载并安装Acaconda3（将Anaconda路径添加至环境变量） https://www.anaconda.com/download/ 2、修改python版本 conda install python=3.6 //换成所需要的版本 3、创建虚拟环境 虚拟环境名称自定义，我将虚拟环境名称设置为py27，python版本需要指定 conda create –n py27 pyt...
caffe+python3.5的配置
wangyu088的专栏
03-07 1768
今天主要介绍下caffe+python3.5的配置，caffe默认的python2.7,而且兼容性很好。但是前段时间装了tensorflow是支持python3的，所以想把caffe的配置也换成python3。强迫症患者..... 打开Makefile.config 去掉下面的注释： PYTHON_LIBRARIES := boost_p...
【Python】ubuntu14安装pycaffe环境：python2.7及依赖库
郭老二
08-07 696
1、问题描述 ubuntu14自带的python2.7版本是python2.7.5，在安装第三方python时，出现错误，提示版本低。在bing上搜索源码安装python2.7.16的步骤，后续使用时，又报错，缺少SLL模块： Can't connect to HTTPS URL because the SSL module is not available 2、源码下载 下载最新的python...
Ubuntu16.04+Cuda9.0+Cudnn7.0+python2.7+Caffe
weixin_42287851的博客
05-23 1491
Ubuntu16.04+Cuda9.0+Cudnn7.0+python2.7+Caffe 1.驱动安装 2.Cuda9.0安装 3.Cudnn7.0安装 以上3步见： https://blog.csdn.net/weixin_42287851/article/details/80421821 4.opencv安装 https://blog.csdn.net/weixin_42287851...
Win10+VS2015+python2.7安装cpu版caffe并测试
邹九的个人博客
11-19 666
Ubuntu的官方安装方法Caffe | Installation: Ubuntu Windows安装方法Caffe | Installation ​ 必须的安装的是Visual Studio 2013 or 2015，本文用到的是VS2015；还有CMake版本需要大于或等于3.4，本文用到的版本是3.12.2 可以使用命令cmake -version查看CMake版本，需要将CMake的bin文件夹加入到环境变量path，CMake的下载地址是：Download | CMake ​ ​ 接着在官.
caffe python3.5 Windows版本编译
ghy_111的博客
03-12 3602
之前编译的是caffe python2.7版本，与caffe python3.5版本还是有差异，记录一下。 下载 caffe windows版本：https://github.com/BVLC/caffe/tree/windows 前提：已安装 anaconda 3（python 3.5），VS2015，CMake，CUDA8.0+cudnn5.1 一caffe-windows\scrip...
Ubuntu 16.04上用系统自带的python3.5.2安装Caffe(CPU only)
jasonxu的博客
07-15 1998
本文为第一次安装caffe，中间费劲曲折，故写下这篇博客让后人少走弯路，主要参考了以下链接：博客：https://blog.csdn.net/muzilinxi90/article/details/53673184     muzilinxi90的博客博客：https://www.linuxidc.com/Linux/2016-09/135034.htm 来源：Linux社区 作者：LinuxCA...
windows配置python2.7和python3.7(绿色免安装版)
用余生去守护的博客
06-15 3970
现在我们虽然痛苦，却依然要快乐，并相信未来。
虚拟机ubuntu18安装caffe python2.7 cpu
最新发布
04-14
我可以为您提供一些关于虚拟机Ubuntu18安装Caffe Python2.7 CPU的指导。首先，在安装Caffe之前，您需要安装必要的依赖项。您可以使用以下命令安装它们： sudo apt-get update sudo apt-get install build-essential...

“相关推荐”对你有帮助么？

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值