caffe安装教程2

最新推荐文章于 2023-08-29 15:06:18 发布

nkszjx2

最新推荐文章于 2023-08-29 15:06:18 发布

阅读量289

点赞数 1

分类专栏： caffe linux

本文链接：https://blog.csdn.net/qq_28424679/article/details/84064534

版权

linux 同时被 2 个专栏收录

36 篇文章 1 订阅

订阅专栏

caffe

12 篇文章 0 订阅

订阅专栏

0.安装一些基本依赖项：
$ sudo apt-get install libprotobuf-dev libleveldb-dev libsnappy-dev libopencv-dev libhdf5-serial-dev protobuf-compiler
$ sudo apt-get install --no-install-recommends libboost-all-dev
$ sudo apt-get install libopenblas-dev liblapack-dev libatlas-base-dev
$ sudo apt-get install libgflags-dev libgoogle-glog-dev liblmdb-dev

一、安装必要的依赖库
(1)安装 matio:
安装方法1：
sudo apt-get install libmatio-dev

正在读取软件包列表... 完成
正在分析软件包的依赖关系树
正在读取状态信息... 完成
将会安装下列额外的软件包：
libmatio2
下列【新】软件包将被安装：
libmatio-dev libmatio2
升级了 0 个软件包，新安装了 2 个软件包，要卸载 0 个软件包，有 2 个软件包未被升级。
需要下载 151 kB 的软件包。
解压缩后会消耗掉 691 kB 的额外空间。
您希望继续执行吗？ [Y/n] y
获取：1 http://cn.archive.ubuntu.com/ubuntu/ trusty/universe libmatio2 amd64 1.5.2-1ubuntu1 [76.8 kB]
获取：2 http://cn.archive.ubuntu.com/ubuntu/ trusty/universe libmatio-dev amd64 1.5.2-1ubuntu1 [73.7 kB]
下载 151 kB，耗时 1秒 (107 kB/s)
正在选中未选择的软件包 libmatio2:amd64。
(正在读取数据库 ... 系统当前共安装有 258518 个文件和目录。)
正准备解包 .../libmatio2_1.5.2-1ubuntu1_amd64.deb ...
正在解包 libmatio2:amd64 (1.5.2-1ubuntu1) ...
正在选中未选择的软件包 libmatio-dev:amd64。
正准备解包 .../libmatio-dev_1.5.2-1ubuntu1_amd64.deb ...
正在解包 libmatio-dev:amd64 (1.5.2-1ubuntu1) ...
正在设置 libmatio2:amd64 (1.5.2-1ubuntu1) ...
正在设置 libmatio-dev:amd64 (1.5.2-1ubuntu1) ...
正在处理用于 libc-bin (2.19-0ubuntu6.9) 的触发器 ...
/sbin/ldconfig.real: /usr/lib/nvidia-375/libEGL.so.1 is not a symbolic link

/sbin/ldconfig.real: /usr/lib32/nvidia-375/libEGL.so.1 is not a symbolic link

##########################################################################

安装方法2：
下载matio(https://sourceforge.net/projects/matio/files/matio/1.5.2/)
tar zxf matio-1.5.2.tar.gz
cd matio-1.5.2
./configure
make
make check
make install
sudo ldconfig

(2)安装 wget
sudo pip install wget

gjh@gjh-System-Product-Name:~$ sudo pip install wget
Collecting wget
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:318: SNIMissingWarning: An HTTPS request has been made, but the SNI (Subject Name Indication) extension to TLS is not available on this platform. This may cause the server to present an incorrect TLS certificate, which can cause validation failures. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#snimissingwarning.
SNIMissingWarning
/usr/local/lib/python2.7/dist-packages/pip/_vendor/requests/packages/urllib3/util/ssl_.py:122: InsecurePlatformWarning: A true SSLContext object is not available. This prevents urllib3 from configuring SSL appropriately and may cause certain SSL connections to fail. You can upgrade to a newer version of Python to solve this. For more information, see https://urllib3.readthedocs.io/en/latest/security.html#insecureplatformwarning.
InsecurePlatformWarning
Downloading wget-3.2.zip
Building wheels for collected packages: wget
Running setup.py bdist_wheel for wget ... done
Stored in directory: /home/gjh/.cache/pip/wheels/6d/98/29/61ccc41148f871009126c2e844e26f73eeb25e12cca92228a5
Successfully built wget
Installing collected packages: wget
Successfully installed wget-3.2

若出错了则按照下面的命令成功：
pip install --upgrade pip --user
pip install --upgrade setuptools --user
sudo pip install wget

二、下载Deeplabv2并编译
1、下载代码：
git clone https://github.com/xmojiao/deeplab_v2.git
（试过许多Deeplab代码，这个最容易编译成功，所以我用的是这个代码编译的）。
2、对 caffe 进行编译：
修改deeplab_v2/deeplab-public-ver2/路径下的 Makefile.config.example文件，重命名为Makefile.config，接着修改这个文件中的内容，将第四行的 “# USE_CUDNN := 1”的 # 去掉。如果需要，因为我用的pycaffe编译，所以不需要修改python的路径，保存退出。
下面为编译 caffe的命令：
cd ~/Desktop/deeplab_v2/deeplab-public-ver2
make clean
make all -j8
make pycaffe
make pytest

错误问题1：运行make pycaffe 时出现 ImportError: No module named google.protobuf.internal
解决方案： sudo pip install protobuf

gjh@gjh-System-Product-Name:~/cell_images/deeplab-v2-master$ pip install protobuf
The directory '/home/gjh/.cache/pip/http' or its parent directory is not owned by the current user and the cache has been disabled. Please check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
The directory '/home/gjh/.cache/pip' or its parent directory is not owned by the current user and caching wheels has been disabled. check the permissions and owner of that directory. If executing pip with sudo, you may want sudo's -H flag.
Collecting protobuf
Downloading protobuf-3.5.0.post1-cp27-cp27mu-manylinux1_x86_64.whl (6.4MB)
100% |████████████████████████████████| 6.4MB 242kB/s
Requirement already satisfied: six>=1.9 in /home/gjh/miniconda2/lib/python2.7/site-packages (from protobuf)
Requirement already satisfied: setuptools in /home/gjh/miniconda2/lib/python2.7/site-packages (from protobuf)
Installing collected packages: protobuf
Successfully installed protobuf-3.5.0.post1

错误问题2：运行make pytest 时出现：
......
I1222 21:38:29.254962 26335 net.cpp:100] Creating Layer conv
I1222 21:38:29.254967 26335 net.cpp:434] conv <- data
I1222 21:38:29.254973 26335 net.cpp:408] conv -> conv
F1222 21:38:31.349524 26335 cudnn.hpp:113] Check failed: status == CUDNN_STATUS_SUCCESS (3 vs. 0) CUDNN_STATUS_BAD_PARAM
*** Check failure stack trace: ***
Aborted (core dumped)
make: *** [pytest] 错误 134
解决方案：选择“# USE_CUDNN := 1”去编译caffe (网上说是因为内存不足等原因，所以我就直接不用CUDNN了，)
cd ~/Desktop/deeplab_v2/deeplab-public-ver2
make clean
make all -j8
make pycaffe
make pytest
运行无误，结束！

三：依次建立存放设置文件夹，预测结果文件夹，数据集txt文件夹，log文件夹，model文件夹，evaluation文件夹
mkdir -p ~/home/gjh/cell_images/exper/voc12/config/deeplab_largeFOV
mkdir -p ~/home/gjh/cell_images/exper/voc12/features/labels
mkdir -p ~/home/gjh/cell_images/exper/voc12/features2/labels
mkdir -p ~/home/gjh/cell_images/exper/voc12/list
mkdir -p ~/home/gjh/cell_images/exper/voc12/log
mkdir -p ~/home/gjh/cell_images/exper/voc12/model/deeplab_largeFOV
mkdir -p ~/home/gjh/cell_images/exper/voc12/res
(2)数据转换
因为pascal voc2012增强数据集的label是mat格式的文件，要把mat格式的label转为png格式的图片.
cd ~/cell_images/exper/voc2012
mkdir cls_png

gjh@gjh-System-Product-Name:~/cell_images/exper/voc2012$ ./mat2png.py /home/gjh/cell_images/cls /home/gjh/cell_images/cls_png
因为pascal voc2012原始数据集的label为三通道RGB图像，但是caffe最后一层softmax loss
层只能识别一通道的label,所以此处我们需要对原始数据集的label进行降维
cd ~~/cell_images/exper/voc2012
mkdir SegmentationClass_1D
./convert_labels.py /home/gjh/cell_images/SegmentationClass/ /home/gjh/cell_images/trainval.txt /home/gjh/cell_images/SegmentationClass_1D/

（3）数据融合
此时我们已经处理好好pascal voc2012 增强数据集和pascal voc2012的原始数据集，为了便于train.txt等文件的调用，我们需要将两个文件夹数据合并到同一个文件中.目前已有数据文件如下：

1. ~/DL_dataset/VOC2012_orig 为原始pascal voc2012文件夹
images数据集的文件名为：JPEGImages
labels数据集文件名为：SegmentationClass_1D

2. ~/DL_dataset/VOC_aug/dataset为pascal voc2012增强数据集文件夹
images数据集的文件名为：img
，jpg图片数为5073
labels数据集文件名为：cls_png，png图片数11355

现分别pascal voc2012增强数据集里的images和labels复制到增强数据集中，若重复则覆盖，合将并数据集的操作如下：

cp /media/gjh/文档/VOC2007-2012数据集/VOC2012/VOCdevkit/VOC2012/SegmentationClass_1D/* /media/gjh/文档/VOC2007-2012数据集/PASCAL_VOC2012/PASCAL_VOC2012/cls_png

cp /media/gjh/文档/VOC2007-2012数据集/VOC2012/VOCdevkit/VOC2012/JPEGImages/* /media/gjh/文档/VOC2007-2012数据集/PASCAL_VOC2012/PASCAL_VOC2012/img/

(4)文件名修改
对应train.txt文件的数据集文件名，修改文件名。

cd ~/DL_dataset/VOC_aug/dataset
mv ./img ./JPEGImages
mv ./cls_png ./SegmentationClassAug

到此处， ~/DL_dataset/VOC_aug/dataset文件夹中
- images数据集的文件名为：JPEGImages ,jpg图片数由5073变为17125
- labels数据集文件名为：cls_png ，png图片数由11355变为12031

（5）修改并运行 run_pascal.sh
进入/Desktop/deeplab_v2/voc2012，修改 run_pascal.sh 文件，主要是修改路径，我的修改后的文件如下：

#!/bin/sh

## MODIFY PATH for YOUR SETTING
ROOT_DIR=/home/mmt/Desktop/my_dataset

CAFFE_DIR=/home/mmt/Desktop/deeplab_v2/deeplab-public-ver2
CAFFE_BIN=${CAFFE_DIR}/build/tools/caffe.bin
........

## Run

RUN_TRAIN=1 #1时train
RUN_TEST=0 #1时test
RUN_TRAIN2=0
RUN_TEST2=0

........

接下来运行代码：
Train：
~/Desktop/deeplab_v2/voc2012
sh ./run_pascal.sh

Test：
将相应变量改为1：
~/Desktop/deeplab_v2/voc2012
sh ./run_pascal_test.sh
运行结果如下：

因为结果保存的是mat文件，如果想转换成png的形式，运行：
cd ~/Desktop/deeplab_v2/voc2012
修改create_labels_21.py的路径，然后此目录运行：
python create_labels_21.py
因为训练一会，我就暂停了，所以test的结果不好，而且图像的分割后的尺寸变了，不知道什么原因，不过经过densecrf后会变回来。

（6）修改并运行 run_densecrf.sh
1）首先对densecrf进行编译。
cd ~/Desktop/deeplab_v2/deeplab-public-ver2/densecrf/
make
有许多warning，但是没出错，不用管。
2）数据整理
因为densecrf只识别ppm格式的图像，所以要转换格式。进入/Desktop/deeplab_v2/deeplab-public-ver2/densecrf/my_script，里面有自带的修改ppm 的MATLAB程序，修改路径，直接运行即可。
代码如下：
#############################################################################################
% save jpg images as bin file for cpp
%
is_server = 1;

dataset = 'voc2012'; %'coco', 'voc2012'

if is_server
if strcmp(dataset, 'voc2012')
img_folder = '/media/gjh/文档/VOC2007-2012datasaet/dataset/JPEGImages';
save_folder = '/media/gjh/文档/VOC2007-2012datasaet/dataset/PPMImages';
elseif strcmp(dataset, 'coco')
img_folder = '/rmt/data/coco/JPEGImages';
save_folder = '/rmt/data/coco/PPMImages';
end
else
img_folder = '../img';
save_folder = '../img_ppm';
end

if ~exist(save_folder, 'dir')
mkdir(save_folder);
end

img_dir = dir(fullfile(img_folder, '*.jpg'));

for i = 1 : numel(img_dir)
% fprintf(1, 'processing %d (%d)...\n', i, numel(img_dir));
img = imread(fullfile(img_folder, img_dir(i).name));

img_fn = img_dir(i).name(1:end-4);
save_fn = fullfile(save_folder, [img_fn, '.ppm']);

imwrite(img, save_fn);
end
#############################################################################################

2）接下来，修改 run_densecrf.sh，注意把 MODEL_NAME=deeplab_largeFOV修改了，原文件少了一个 p。
#############################################################################################

#!/bin/bash

###########################################
# You can either use this script to generate the DenseCRF post-processed results
# or use the densecrf_layer (wrapper) in Caffe
###########################################
DATASET=voc2012
LOAD_MAT_FILE=1

MODEL_NAME=deeplab_largeFOV

TEST_SET=val #val, test

# the features folder save the features computed via the model trained with the train set
# the features2 folder save the features computed via the model trained with the trainval set
FEATURE_NAME=features #features, features2
FEATURE_TYPE=fc8

# specify the parameters
MAX_ITER=10

Bi_W=4
Bi_X_STD=49
Bi_Y_STD=49
Bi_R_STD=5
Bi_G_STD=5
Bi_B_STD=5

POS_W=3
POS_X_STD=3
POS_Y_STD=3

#######################################
# MODIFY THE PATY FOR YOUR SETTING
#######################################
SAVE_DIR=/media/gjh/文档/VOC2007-2012datasaet/dataset/${DATASET}/res/${FEATURE_NAME}/${MODEL_NAME}/${TEST_SET}/${FEATURE_TYPE}/post_densecrf_W${Bi_W}_XStd${Bi_X_STD}_RStd${Bi_R_STD}_PosW${POS_W}_PosXStd${POS_X_STD}

echo "SAVE TO ${SAVE_DIR}"

CRF_DIR=/home/gjh/cell_images/deeplab-v2-master/densecrf
IMG_DIR_NAME=dataset

# if [ ${DATASET} == "voc12" ]
# then
# IMG_DIR_NAME=pascal/VOCdevkit/VOC2012
# elif [ ${DATASET} == "coco" ]
# then
# IMG_DIR_NAME=coco
# elif [ ${DATASET} == "voc10_part" ]
# then
# IMG_DIR_NAME=pascal/VOCdevkit/VOC2012
# fi

# NOTE THAT the densecrf code only loads ppm images
IMG_DIR=/media/gjh/文档/VOC2007-2012datasaet/${IMG_DIR_NAME}/PPMImages

# if [ ${LOAD_MAT_FILE} == 1 ]
# then
# the features are saved in .mat format
CRF_BIN=${CRF_DIR}/prog_refine_pascal_v4
FEATURE_DIR=/media/gjh/文档/VOC2007-2012datasaet/dataset/${DATASET}/${FEATURE_NAME}/${MODEL_NAME}/${TEST_SET}/${FEATURE_TYPE}
# else
# the features are saved in .bin format (has called SaveMatAsBin.m in the densecrf/my_script)
# CRF_BIN=${CRF_DIR}/prog_refine_pascal
# FEATURE_DIR=/rmt/work/deeplab/exper/${DATASET}/${FEATURE_NAME}/${MODEL_NAME}/${TEST_SET}/${FEATURE_TYPE}/bin
# fi

mkdir -p ${SAVE_DIR}

# run the program
${CRF_BIN} -id ${IMG_DIR} -fd ${FEATURE_DIR} -sd ${SAVE_DIR} -i ${MAX_ITER} -px ${POS_X_STD} -py ${POS_Y_STD} -pw ${POS_W} -bx ${Bi_X_STD} -by ${Bi_Y_STD} -br ${Bi_R_STD} -bg ${Bi_G_STD} -bb ${Bi_B_STD} -bw ${Bi_W}

##################################################################################################################################
运行结果：
gjh@gjh-System-Product-Name:~/cell_images/deeplab-v2-master/voc2012$ sh ./run_densecrf.sh
SAVE TO /media/gjh/文档/VOC2007-2012datasaet/dataset/voc2012/res/features/deeplab_largeFOV/val/fc8/post_densecrf_W4_XStd49_RStd5_PosW3_PosXStd3
Input Parameters:
ImgDir: /media/gjh/文档/VOC2007-2012datasaet/dataset/PPMImages
FeatureDir: /media/gjh/文档/VOC2007-2012datasaet/dataset/voc2012/features/deeplab_largeFOV/val/fc8
SaveDir: /media/gjh/文档/VOC2007-2012datasaet/dataset/voc2012/res/features/deeplab_largeFOV/val/fc8/post_densecrf_W4_XStd49_RStd5_PosW3_PosXStd3
MaxIterations: 10
PosW: 3
PosXStd: 3
PosYStd: 3
Bi_W: 4
Bi_X_Std: 49
Bi_Y_Std: 49
Bi_R_Std: 5
Bi_G_Std: 5
Bi_B_Std: 5

3）然后运行 /home/mmt/crf/deeplab-public-ver2/densecrf/my_script/GetDenseCRFResult.m把bin生成图片格式
注意修改文件路径（GetDenseCRFResult.m，SetupEnv在/deeplab_v2/deeplab-public-ver2/matlab/my_script中），
两个程序的代码如下：
##################################################################################################################################
% compute the densecrf result (.bin) to png
%

addpath('/home/gjh/cell_images/deeplab-v2-master/matlab/my_script');
% SetupEnv在/home/gjh/cell_images/deeplab-v2-master/matlab/my_script中
SetupEnv;

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% You do not need to change values below
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
if is_server
if learn_crf
post_folder = sprintf('post_densecrf_W%d_XStd%d_RStd%d_PosW%d_PosXStd%d_ModelType%d_Epoch%d', bi_w, bi_x_std, bi_r_std, pos_w, pos_x_std, model_type, epoch);

map_folder = fullfile('/media/gjh/文档/VOC2007-2012datasaet/dataset', dataset, 'densecrf', 'res', feature_name, model_name, testset, feature_type, post_folder);

save_root_folder = fullfile('/media/gjh/文档/VOC2007-2012datasaet/dataset', dataset, 'res', feature_name, model_name, testset, feature_type, post_folder); ;
else
post_folder = sprintf('post_densecrf_W%d_XStd%d_RStd%d_PosW%d_PosXStd%d', bi_w, bi_x_std, bi_r_std, pos_w, pos_x_std);

map_folder = fullfile('/media/gjh/文档/VOC2007-2012datasaet/dataset', dataset, 'res', feature_name, model_name, testset, feature_type, post_folder);

save_root_folder = map_folder;
end
else
map_folder = '../result';
end

map_dir = dir(fullfile(map_folder, '*.bin'));

fprintf(1,' saving to %s\n', save_root_folder);

if strcmp(dataset, 'voc2012')
seg_res_dir = [save_root_folder '/results/VOC2012/'];
elseif strcmp(dataset, 'coco')
seg_res_dir = [save_root_folder, '/results/COCO2014/'];
else
error('Wrong dataset!');
end

save_result_folder = fullfile(seg_res_dir, 'Segmentation', [id '_' testset '_cls']);

if ~exist(save_result_folder, 'dir')
mkdir(save_result_folder);
end

for i = 1 : numel(map_dir)
fprintf(1, 'processing %d (%d)...\n', i, numel(map_dir));
map = LoadBinFile(fullfile(map_folder, map_dir(i).name), 'int16');

img_fn = map_dir(i).name(1:end-4);
imwrite(uint8(map), colormap, fullfile(save_result_folder, [img_fn, '.png']));
end
##################################################################################################################################
##################################################################################################################################
SetupEnv.m
##################################################################################################################################
% set up the environment variables
%

clear all; close all;
load('./pascal_seg_colormap.mat');

is_server = 1;

crf_load_mat = 1; % the densecrf code load MAT files directly (no call SaveMatAsBin.m)
% used ONLY by DownSampleFeature.m
learn_crf = 0; % NOT USED. Set to 0

is_mat = 1; % the results to be evaluated are saved as mat (1) or png (0)
has_postprocess = 0; % has done densecrf post processing (1) or not (0)
is_argmax = 0; % the output has been taken argmax already (e.g., coco dataset).
% assume the argmax takes C-convention (i.e., start from 0)

debug = 0; % if debug, show some results

% vgg128_noup (not optimized well), aka DeepLab
% bi_w = 5, bi_x_std = 50, bi_r_std = 10

% vgg128_ms_pool3, aka DeepLab-MSc
% bi_w = 3, bi_x_std = 95, bi_r_std = 3

% vgg128_noup_pool3_cocomix, aka DeepLab-COCO
% bi_w = 5, bi_x_std = 67, bi_r_std = 3

%% these are used for the bounding box weak annotation experiments (i.e., to generate the Bbox-Seg)
% erode_gt (bbox)
% bi_w = 41, bi_x_std = 33, bi_r_std = 4

% erode_gt/bboxErode20
% bi_w = 45, bi_x_std = 37, bi_r_std = 3, pos_w = 15, pos_x_std = 3

%
% initial or default values for crf
% these parameter should be the same with the document 'run_densecrf.sh'
bi_w = 4;
bi_x_std = 49;
bi_r_std = 5;

pos_w = 3;
pos_x_std = 3;

%
dataset = 'voc2012'; %'voc2012', 'coco' % 修改
trainset = 'train_aug'; % not used
testset = 'val'; %'val', 'test'

model_name = 'deeplab_largeFOV'; % 修改

feature_name = 'features';
feature_type = 'fc8'; % fc8 / crf

id = 'comp6';

%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%% used for cross-validation
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
rng(10)

% downsampling files for cross-validation
down_sample_method = 2; % 1: equally sample with "down_sample_rate", 2: randomly pick "num_sample" samples
down_sample_rate = 8;
num_sample = 100; % number of samples used for cross-validation

% ranges for cross-validation
range_pos_w = [3];
range_pos_x_std = [3];

range_bi_w = [5];
range_bi_x_std = [49];
range_bi_r_std = [4 5];
##################################################################################################################################
4)至此，deeplabv2 程序已调试完。

nkszjx2

关注

1
点赞
踩
0

收藏

觉得还不错? 一键收藏
打赏
0
评论
caffe安装教程2

0.安装一些基本依赖项：$ sudo apt-get install libprotobuf-dev libleveldb-dev libsnappy-dev libopencv-dev libhdf5-serial-dev protobuf-compiler $ sudo apt-get install --no-install-recommends libboost-all-dev$...
复制链接

扫一扫