这是caffe官方文档Notebook Examples中的第四个例子, 链接地址：http://nbviewer.jupyter.org/github/BVLC/caffe/blob/master/examples/net_surgery.ipynb
这个例子讲述如何编辑caffe的模型参数以满足特定的需要，所有的网络数据，残差，参数都在pycaffe中。
与之前的分类问题不同，本例最后的输出不是对整个图分类概率，而是输出图的每块区域属于哪一类，本例中，用451*451的图像作为输入，输出是一个8*8的分类图，图中的数字表示所代表的区域属于那个类别，例如：
[[282 282 281 281 281 281 277 282]
[281 283 283 281 281 281 281 282]
[283 283 283 283 283 283 287 282]
[283 283 283 281 283 283 283 259]
[283 283 283 283 283 283 283 259]
[283 283 283 283 283 283 259 259]
[283 283 283 283 259 259 259 277]
[335 335 283 259 263 263 263 277]]
这里，282是虎猫，281是斑猫，283是波斯猫
1. 导入包python，更改路径
import numpy as np
import matplotlib.pyplot as plt
#%matplotlib inline
import Image
caffe_root = '/home/sindyz/caffe-master/'
import sys
sys.path.insert(0, caffe_root+'python')
import caffe
# configure plotting
plt.rcParams['figure.figsize'] = (10, 10)
plt.rcParams['image.interpolation'] = 'nearest'
plt.rcParams['image.cmap'] = 'gray'
model_file = caffe_root + 'examples/net_surgery/conv.prototxt'
image_file = caffe_root + 'examples/images/cat_gray.jpg'为了展示如何载入，操作，保存参数。这里在一个简单的网络中设计我们自己的滤波器，这个网络只有一个卷积层，两个blobs，data是输入数据，conv为卷积输出，参数conv是卷积滤波器的权重和偏移。
# Load the net, list its data and params, and filter an example image.
caffe.set_mode_cpu()
net = caffe.Net(model_file, caffe.TEST)
print("blobs {}\nparams {}".format(net.blobs.keys(), net.params.keys()))
# load image and prepare as a single input batch for Caffe
im = np.array(Image.open(image_file))
plt.title("original image")
plt.imshow(im)
plt.axis('off')
im_input=im[np.newaxis, np.newaxis,:,:]
net.blobs['data'].reshape(*im_input.shape)
net.blobs['data'].data[...] = im_inputblobs ['data', 'conv'] params ['conv']
网络结构：
# Simple single-layer network to showcase editing model parameters.
name: "convolution"
input: "data"
input_shape {
dim: 1
dim: 1
dim: 100
dim: 100
}
layer {
name: "conv"
type: "Convolution"
bottom: "data"
top: "conv"
convolution_param {
num_output: 3
kernel_size: 5
stride: 1
weight_filler {
type: "gaussian"
std: 0.01
}
bias_filler {
type: "constant"
value: 0
}
}
}3. 显示滤波结果
# helper show filter outputs
def show_filters(net):
net.forward()
plt.figure()
filt_min, filt_max = net.blobs['conv'].data.min(), net.blobs['conv'].data.max()
for i in range(3):
plt.subplot(1,4,i+2)
plt.title("filter #{} output".format(i))
plt.imshow(net.blobs['conv'].data[0, i], vmin=filt_min, vmax=filt_max)
plt.tight_layout()
plt.axis('off')
# filter the image with initial
show_filters(net)
4. 提高滤波器的偏置将相应的提高他的输出
conv0 = net.blobs['conv'].data[0, 0]
print("pre-surgery output mean {:.2f}".format(conv0.mean()))
# set first filter bias to 10
net.params['conv'][1].data[0] = 1.
net.forward()
print("post-surgery output mean {:.2f}".format(conv0.mean()))pre-surgery output mean -12.93 post-surgery output mean -11.93
5. 更改滤波器
也可以更改滤波器，我们可以使用任意的核如高斯模糊滤波，Sobel边缘算子等等。在下面的改动中，第０个滤波器为高斯模糊滤波，第１个滤波器和第二个滤波器为水平和数值方向的Sobel滤波算子。
ksize = net.params['conv'][0].data.shape[2:]
# make Gaussian blur
sigma = 1.http://write.blog.csdn.net/postedit/50681609
y, x = np.mgrid[-ksize[0]//2 + 1:ksize[0]//2 + 1, -ksize[1]//2 + 1:ksize[1]//2 + 1]
g = np.exp(-((x**2 + y**2)/(2.0*sigma**2)))
gaussian = (g / g.sum()).astype(np.float32)
net.params['conv'][0].data[0] = gaussian
# make Sobel operator for edge detection
net.params['conv'][0].data[1:] = 0.
sobel = np.array((-1, -2, -1, 0, 0, 0, 1, 2, 1), dtype=np.float32).reshape((3,3))
net.params['conv'][0].data[1, 0, 1:-1, 1:-1] = sobel # horizontal
net.params['conv'][0].data[2, 0, 1:-1, 1:-1] = sobel.T # vertical
show_filters(net)
可以看出，第０张图片是模糊的，第１张图片选出了水平方向的边缘，第２张图片选出了垂直方向的边缘
6. 在全连接网络中构造分类器
现在将caffe中自带的ImageNet模型“caffenet”转换成一个全卷积网络，以便于对大量输入高效，密集的运算。这个模型产生一个与输入相同大小的分类图而不是单一的分类器。特别的，在451x451的输入中，一个8x8的分类图提供了64倍的输出但是仅消耗了3倍的时间。The computation exploits a natural efficiency of convolutional network (convnet) structure by amortizing the computation of overlapping receptive fields.
我们将CaffeNet的InnerProduct矩阵乘法层转换为卷积层，这是仅有的改变: the other layer types are agnostic to spatial size. 卷积是平移不变的, 激活是元素操作。 fc6全连接层用fc6-conv替换，用6*6滤波器已间隔为1对pool5的输出进行滤波。回到图像空间中，对每个227*227的输入且步长为32的图像给定一个分类器, 输出图像和感受野的尺寸相同； output = (input -kernel_size) / stride + 1。
执行：
!diff examples/net_surgery/bvlc_caffenet_full_conv.prototxt models/bvlc_reference_caffenet/deploy.prototxt
1,2c1
< # Fully convolutional network version of CaffeNet.
< name: "CaffeNetConv"
---
> name: "CaffeNet"
5c4
< dim: 1
---
> dim: 10
7,8c6,7
< dim: 451
< dim: 451
---
> dim: 227
> dim: 227
154,155c153,154 #做过改变的行
< name: "fc6-conv" #将fc6改为fc6-conv
< type: "Convolution" #内积层变为卷基层
---
> name: "fc6"
> type: "InnerProduct"
157,158c156,157
< top: "fc6-conv"
< convolution_param {#变为卷积层后需要设置参数
---
> top: "fc6"
> inner_product_param {
160d158
< kernel_size: 6 #pool5的输出是36,全连接卷积层为6*6
166,167c164,165
< bottom: "fc6-conv"
< top: "fc6-conv"
---
> bottom: "fc6"
> top: "fc6"
172,173c170,171
< bottom: "fc6-conv"
< top: "fc6-conv"
---
> bottom: "fc6"
> top: "fc6"
179,183c177,181
< name: "fc7-conv"
< type: "Convolution"
< bottom: "fc6-conv"
< top: "fc7-conv"
< convolution_param {
---
> name: "fc7"
> type: "InnerProduct"
> bottom: "fc6"
> top: "fc7"
> inner_product_param {
185d182
< kernel_size: 1
191,192c188,189
< bottom: "fc7-conv"
< top: "fc7-conv"
---
> bottom: "fc7"
> top: "fc7"
197,198c194,195
< bottom: "fc7-conv"
< top: "fc7-conv"
---
> bottom: "fc7"
> top: "fc7"
204,208c201,205
< name: "fc8-conv"
< type: "Convolution"
< bottom: "fc7-conv"
< top: "fc8-conv"
< convolution_param {
---
> name: "fc8"
> type: "InnerProduct"
> bottom: "fc7"
> top: "fc8"
> inner_product_param {
210d206
< kernel_size: 1
216c212
< bottom: "fc8-conv"
---
> bottom: "fc8"
可见，结构上唯一需要改变的就是将全连接的分类器内积层改为卷积层，并且使用6*6的滤波器，因为参考模型的分类以pool6的36个输出作为fc6-conv的输入。为了保证密集分类，令步长为1。注意，重命名是为了避免当模型命名为"预训练"模型时caffe取载入旧的参数。
#载入传统的网络模型获取全连接层的参数
net = caffe.Net('models/bvlc_reference_caffenet/deploy.prototxt',
'models/bvlc_reference_caffenet/bvlc_reference_caffenet.caffemodel',
caffe.TEST)
params = ['fc6', 'fc7', 'fc8']
fc_params = {pr: (net.params[pr][0].data, net.params[pr][1].data) for pr in params}
for fc in params:
print '{} weights are {} dimensional and biases are {} dimensional'.format(fc, fc_params[fc][0].shape, fc_params[fc][1].shape)输出：
fc6 weights are (4096, 9216) dimensional and biases are (4096,) dimensional
fc7 weights are (4096, 4096) dimensional and biases are (4096,) dimensional
fc8 weights are (1000, 4096) dimensional and biases are (1000,) dimensional
考虑到内部产生的参数的形状，权值的规模是输入*输出，偏置的规模是输出的规模。
# 导入全卷积网去移植的参数
net_full_conv = caffe.Net('examples/net_surgery/bvlc_caffenet_full_conv.prototxt',
'models/bvlc_reference_caffenet/bvlc_reference_caffenet.caffemodel',
caffe.TEST)
params_full_conv = ['fc6-conv', 'fc7-conv', 'fc8-conv']
# conv_params = {name: (weights, biases)}
conv_params = {pr: (net_full_conv.params[pr][0].data, net_full_conv.params[pr][1].data) for pr in params_full_conv}
for conv in params_full_conv:
print '{} weights are {} dimensional and biases are {} dimensional'.format(conv, conv_params[conv][0].shape, conv_params[conv][1].shape)输出：
fc6-conv weights are (4096, 256, 6, 6) dimensional and biases are (4096,) dimensional
fc7-conv weights are (4096, 4096, 1, 1) dimensional and biases are (4096,) dimensional
fc8-conv weights are (1000, 4096, 1, 1) dimensional and biases are (1000,) dimensional
卷积权重由 output*input*heigth*width的规模决定，为了将内部产生的权重对应到卷积滤波器中，需要将内部产生的权值转变为channel*height*width规模的滤波矩阵。但是他们全部在内存中(按行存储), 所以我们可以直接指定，即两者是一致的。
偏置与内连接层相同。
开始移植；
for pr, pr_conv in zip(params, params_full_conv):
conv_params[pr_conv][0].flat = fc_params[pr][0].flat # flat unrolls the arrays
conv_params[pr_conv][1][...] = fc_params[pr][1]存储新的模型：
net_full_conv.save('examples/net_surgery/bvlc_caffenet_full_conv.caffemodel')im = caffe.io.load_image('examples/images/cat.jpg')
transformer = caffe.io.Transformer({'data': net_full_conv.blobs['data'].data.shape})
transformer.set_mean('data', np.load('python/caffe/imagenet/ilsvrc_2012_mean.npy').mean(1).mean(1))
transformer.set_transpose('data', (2,0,1))
transformer.set_channel_swap('data', (2,1,0))
transformer.set_raw_scale('data', 255.0)
# make classification map by forward and print prediction indices at each location
out = net_full_conv.forward_all(data=np.asarray([transformer.preprocess('data', im)]))
print out['prob'][0].argmax(axis=0)
# show net input and confidence map (probability of the top prediction at each location)
plt.subplot(1, 2, 1)
plt.imshow(transformer.deprocess('data', net_full_conv.blobs['data'].data[0]))
plt.subplot(1, 2, 2)
plt.imshow(out['prob'][0,281])输出：
[[282 282 281 281 281 281 277 282]
[281 283 283 281 281 281 281 282]
[283 283 283 283 283 283 287 282]
[283 283 283 281 283 283 283 259]
[283 283 283 283 283 283 283 259]
[283 283 283 283 283 283 259 259]
[283 283 283 283 259 259 259 277]
[335 335 283 259 263 263 263 277]]
分类结果包含各种猫：282是虎猫，281是斑猫，283是波斯猫，还有狐狸及其他哺乳动物。
用这种方式，全连接网络可以用于提取图像的密集特征，这比分类图本身更加有用。
注意，这种模型不完全适用与滑动窗口检测，因为它是为了整张图片的分类而训练的，然而，滑动窗口训练和微调可以通过在真值和loss的基础上定义一个滑动窗口，这样一个loss图就会由每一个位置组成，由此可以像往常一样解决。
参考资料：
http://nbviewer.jupyter.org/github/BVLC/caffe/blob/master/examples/net_surgery.ipynb
caffe新手，希望大家多多交流，共同进步。
942
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
