caffe+unbuntu14.04+GPU运行srcnn

本文链接：https://blog.csdn.net/m0_37407756/article/details/70545777

caffe+unbuntu14.04+GPU运行srcnn：

因为最近看了一篇论文，SRCNN-Image Super-Resolution Using Deep Convolutional Networks

想实现一下，把其中的过程记录一下，希望能对后者有用，因本人也是初学者，整理的内容都是自己遇到的实际错误，这期间难免会有差错，还请批评指正。

1.准备工作：

在配置好的caffe文件夹中，进入examples目录，创建SRCNN文件夹，并进入文件夹，再创建modelTest Train四个文件夹。

model：用来保存驯良好的模型

Test：保存要测试的图片

Train：保存要训练的图片

2.产生训练和测试数据

1)在SRCNN文件夹下面(以下均是在此文件夹下)创建两个路径文件。

$vim train.txt

输入内容：

examples/SRCNN/train.h5

:wq保存文档。

$vim test.txt

输入内容：

examples/SRCNN/test.h5

:wq保存文档

2)产生训练数据

$vim generate_train.m

输入以下的内容：

clear;closeall;
%%settings
folder= 'Train';
savepath= 'train.h5';
size_input= 33;
size_label= 21;
scale= 3;
stride= 14;
%%initialization
data= zeros(size_input, size_input, 1, 1);
label= zeros(size_label, size_label, 1, 1);
padding= abs(size_input - size_label)/2;
count= 0;
%%generate data
filepaths= dir(fullfile(folder,'*.bmp'));
fori = 1 : length(filepaths)
image= imread(fullfile(folder,filepaths(i).name));
image= rgb2ycbcr(image);
image= im2double(image(:, :, 1));
im_label= modcrop(image, scale);
[hei,wid]= size(im_label);
im_input=imresize(imresize(im_label,1/scale,'bicubic'),[hei,wid],'bicubic');
forx = 1 : stride : hei-size_input+1
fory = 1 :stride : wid-size_input+1
subim_input= im_input(x : x+size_input-1, y : y+size_input-1);
subim_label= im_label(x+padding : x+padding+size_label-1, y+padding :y+padding+size_label-1);
count=count+1;
data(:,:, 1, count) = subim_input;
label(:,:, 1, count) = subim_label;
end
end
end
order= randperm(count);
data= data(:, :, 1, order);
label= label(:, :, 1, order);
%%writing to HDF5
chunksz= 128;
created_flag= false;
totalct= 0;
forbatchno = 1:floor(count/chunksz)
last_read=(batchno-1)*chunksz;
batchdata= data(:,:,1,last_read+1:last_read+chunksz);
batchlabs= label(:,:,1,last_read+1:last_read+chunksz);
startloc= struct('dat',[1,1,1,totalct+1], 'lab', [1,1,1,totalct+1]);
curr_dat_sz= store2hdf5(savepath, batchdata, batchlabs, ~created_flag, startloc,chunksz);
created_flag= true;
totalct= curr_dat_sz(end);
end
h5disp(savepath);

终端下输入：

$matlab -nodesktop -nosplash -logfile generate_train.log -rgenerate_train

产生训练数据train.h5

3)产生测试数据

$vim generate_test.m

输入以下内容：

clear;closeall;
%%settings
folder= 'Test/Set5';
savepath= 'test.h5';
size_input= 33;
size_label= 21;
scale= 3;
stride= 21 ;
%%initialization
data= zeros(size_input, size_input, 1, 1);
label= zeros(size_label, size_label, 1, 1);
padding= abs(size_input - size_label)/2;
count= 0;
%%generate data
filepaths= dir(fullfile(folder,'*.bmp'));
fori = 1 : length(filepaths)
image= imread(fullfile(folder,filepaths(i).name));
image= rgb2ycbcr(image);
image= im2double(image(:, :, 1));
im_label= modcrop(image, scale);
[hei,wid]= size(im_label);
im_input=imresize(imresize(im_label,1/scale,'bicubic'),[hei,wid],'bicubic');
forx = 1 : stride : hei-size_input+1
fory = 1 :stride : wid-size_input+1
subim_input= im_input(x : x+size_input-1, y : y+size_input-1);
subim_label= im_label(x+padding : x+padding+size_label-1, y+padding :y+padding+size_label-1);
count=count+1;
data(:,:, 1, count) = subim_input;
label(:,:, 1, count) = subim_label;
end
end
end
order= randperm(count);
data= data(:, :, 1, order);
label= label(:, :, 1, order);
%%writing to HDF5
chunksz= 2;
created_flag= false;
totalct= 0;
forbatchno = 1:floor(count/chunksz)
last_read=(batchno-1)*chunksz;
batchdata= data(:,:,1,last_read+1:last_read+chunksz);
batchlabs= label(:,:,1,last_read+1:last_read+chunksz);
startloc= struct('dat',[1,1,1,totalct+1], 'lab', [1,1,1,totalct+1]);
curr_dat_sz= store2hdf5(savepath, batchdata, batchlabs, ~created_flag, startloc,chunksz);
created_flag= true;
totalct= curr_dat_sz(end);
end
h5disp(savepath);

终端下输入：

$matlab -nodesktop -nosplash -logfile generate_test.log -rgenerate_test

产生测试数据test.h5。仅仅用于判断训练到达什么地步。

3.建立训练文件

1.)建立solver文件

$vim SRCNN_solver.prototxt

输入内容：

#The train/test net protocol buffer definition
net:"examples/SRCNN/SRCNN_net.prototxt"
test_iter:556
#Carry out testing every 500 training iterations.
test_interval:500
#The base learning rate, momentum and the weight decay of the network.
base_lr:0.0001
momentum:0.9
weight_decay:0
#The learning rate policy
lr_policy:"fixed"
#Display every 100 iterations
display:100
#The maximum number of iterations
max_iter:1500000
#snapshot intermediate results
snapshot:500
snapshot_prefix:"examples/SRCNN/SRCNN"
#solver mode: CPU or GPU
solver_mode:GPU

:wq保存退出。

2.)建立net文件

$vim SRCNN_net.prototxt

在此输入：

name:"SRCNN"
layer{
name:"data"
type:"HDF5Data"
top:"data"
top:"label"
hdf5_data_param{
source:"examples/SRCNN/train.txt"
batch_size:128
}
include:{ phase: TRAIN }
}
layer{
name:"data"
type:"HDF5Data"
top:"data"
top:"label"
hdf5_data_param{
source:"examples/SRCNN/test.txt"
batch_size:2
}
include:{ phase: TEST }
}
layer{
name:"conv1"
type:"Convolution"
bottom:"data"
top:"conv1"
param{
lr_mult:1
}
param{
lr_mult:0.1
}
convolution_param{
num_output:64
kernel_size:9
stride:1
pad:0
weight_filler{
type:"gaussian"
std:0.001
}
bias_filler{
type:"constant"
value:0
}
}
}
layer{
name:"relu1"
type:"ReLU"
bottom:"conv1"
top:"conv1"
}
layer{
name:"conv2"
type:"Convolution"
bottom:"conv1"
top:"conv2"
param{
lr_mult:1
}
param{
lr_mult:0.1
}
convolution_param{
num_output:32
kernel_size:1
stride:1
pad:0
weight_filler{
type:"gaussian"
std:0.001
}
bias_filler{
type:"constant"
value:0
}
}
}
layer{
name:"relu2"
type:"ReLU"
bottom:"conv2"
top:"conv2"
}
layer{
name:"conv3"
type:"Convolution"
bottom:"conv2"
top:"conv3"
param{
lr_mult:0.1
}
param{
lr_mult:0.1
}
convolution_param{
num_output:1
kernel_size:5
stride:1
pad:0
weight_filler{
type:"gaussian"
std:0.001
}
bias_filler{
type:"constant"
value:0
}
}
}
layer{
name:"loss"
type:"EuclideanLoss"
bottom:"conv3"
bottom:"label"
top:"loss"
}

:wq保存退出。

4.SRCNN训练

$vim train.sh

输入：

#!/usr/bin/envsh

/home/abc/caffe-master/build/tools/caffetrain--solver=/home/abc/caffe-master/examples/SRCNN/SRCNN_solver.prototxt#\

#--weights=models/bvlc_reference_caffenet/bvlc_reference_caffenet.caffemodel\

#--gpu0

增加运行权限：

$chmod +x train.sh

运行脚本文件：

$./train.sh

5.保存滤波器

1.)创建mat文件

$cp SRCNN_net.prototxt SRCNN_mat.prototxt

输入以下内容：

name:"SRCNN"
input:"data"
input_dim:1
input_dim:1
input_dim:33
input_dim:33
input:"label"
input_dim:1
input_dim:1
input_dim:21
input_dim:21
layer{
name:"conv1"
type:"Convolution"
bottom:"data"
top:"conv1"
param{
lr_mult:1
}
param{
lr_mult:0.1
}
convolution_param{
num_output:64
kernel_size:9
stride:1
pad:0
weight_filler{
type:"gaussian"
std:0.001
}
bias_filler{
type:"constant"
value:0
}
}
}
layer{
name:"relu1"
type:"ReLU"
bottom:"conv1"
top:"conv1"
}
layer{
name:"conv2"
type:"Convolution"
bottom:"conv1"
top:"conv2"
param{
lr_mult:1
}
param{
lr_mult:0.1
}
convolution_param{
num_output:32
kernel_size:1
stride:1
pad:0
weight_filler{
type:"gaussian"
std:0.001
}
bias_filler{
type:"constant"
value:0
}
}
}
layer{
name:"relu2"
type:"ReLU"
bottom:"conv2"
top:"conv2"
}
layer{
name:"conv3"
type:"Convolution"
bottom:"conv2"
top:"conv3"
param{
lr_mult:0.1
}
param{
lr_mult:0.1
}
convolution_param{
num_output:1
kernel_size:5
stride:1
pad:0
weight_filler{
type:"gaussian"
std:0.001
}
bias_filler{
type:"constant"
value:0
}
}
}
layer{
name:"loss"
type:"EuclideanLoss"
bottom:"conv3"
bottom:"label"
top:"loss"
}

:wq保存即可。

2.)创建M文件，将weights和bias转为.mat格式：

$vim saveFilters.m

%layers中的parameter(1)表示权重参数,parameter(2)表示偏置参数；
caffe.reset_all();%清除已经创建的所有solver和独立net;
clear;close all;
%%settings
folder= 'examples/SRCNN/';
model= [folder 'SRCNN_mat.prototxt'];
weights= [folder 'SRCNN_iter_500.caffemodel'];
savepath= [folder 'x3.mat'];
layers= 3;
%%load model using mat_caffe
net= caffe.Net(model,weights,'test');%创建一个网络load权重weights；
%%reshap parameters
weights_conv= cell(layers,1);
foridx = 1 : layers
conv_filters= net.layers(['conv' num2str(idx)]).params(1).get_data();
[~,fsize,channel,fnum]= size(conv_filters);
ifchannel == 1
weights= single(ones(fsize^2, fnum));
else
weights= single(ones(channel, fsize^2, fnum));
end
fori = 1 : channel
forj = 1 : fnum
temp= conv_filters(:,:,i,j);
ifchannel == 1
weights(:,j)= temp(:);
else
weights(i,:,j)= temp(:);
end
end
end
weights_conv{idx}= weights;
end
%%save parameters
weights_conv1= weights_conv{1};
weights_conv2= weights_conv{2};
weights_conv3= weights_conv{3};
biases_conv1= net.layers('conv1').params(2).get_data();
biases_conv2= net.layers('conv2').params(2).get_data();
biases_conv3= net.layers('conv3').params(2).get_data();
save(savepath,'weights_conv1','biases_conv1','weights_conv2','biases_conv2','weights_conv3','biases_conv3');

:wq保存。