Papers:
Fully Convolutional Models for Semantic Segmentation
Evan Shelhamer*, Jonathan Long*, Trevor Darrell
PAMI 2016
arXiv:1605.06211
Fully Convolutional Models for Semantic Segmentation
Jonathan Long*, Evan Shelhamer*, Trevor Darrell
CVPR 2015
arXiv:1411.4038
官方的Code提供了PASCAL VOC models,SIFT Flow models,PASCAL-Context models的完整(32s,16s,8s)的代码,但对于NYUD只提供了32s的代码,这里我们就以NYUD作为例子说明一下FCN-8s训练的完整过程(网上有很多教程,但不是不完整,就是存在错误)。
源代码下载和数据集预处理
下载官方源代码:
git clone https://github.com/shelhamer/fcn.berkeleyvision.org.git
下载VGG16的预训练模型并放在FCN源码文件夹中的ilsvrc-nets文件夹下:
cd /fcn.berkeleyvision.org/ilsvrc-nets
wget http://www.robots.ox.ac.uk/~vgg/software/very_deep/caffe/VGG_ILSVRC_16_layers.caffemodel
获取与其相对应的deploy文件:
wget https://gist.githubusercontent.com/ksimonyan/211839e770f7b538e2d8/raw/0067c9b32f60362c74f4c445a080beed06b07eb3/VGG_ILSVRC_16_layers_deploy.prototxt
下载数据集:
cd data/nyud/
wget http://people.eecs.berkeley.edu/~sgupta/cvpr13/data.tgz
tar -xvf data.tgz
这个时候,nyud的文件夹应该是这个样子:
其中data文件夹中有三个子文件夹:benchmarkData, colorImage, pointCloud. 其中benchmarkData/groundTruth中储存这所有我们需要的分割的真值,colorImage文件夹储存着原始的RGB文件.由于源代码设置的groundTruth路径和现有的路径不一样,所以我们要把groundTruth文件copy到指定路径:
mkdir segmentation
cp data/benchmarkData/groundTruth/*.mat segmentation/
同时,我们要合并train.txt和val.txt: 在nyud文件夹中新建一个空白的.txt文件并命名为trainval.txt,然后将train.txt和val.txt中的内容Copy过去.这个时候nyud文件夹应为是这个样子:
数据准备完毕,现在开始训练FCN-32s网络.
FCN-32s网络训练:
把要用到的.py文件Copy到nyud-fcn32s-color文件夹:
cd fcn.berkeleyvision.org
cp *.py nyud-fcn32s-color/
cd nyud-fcn32s-color
rm pascalcontext_layers.py
rm voc_helper.py
rm voc_layer.py
rm siftflow_layers.py
修改solver.prototxt文件,下面是我个人的文件内容,关于相关参数的含义请参考:Caffe学习系列(7):solver及其配置 - denny402 - 博客园
train_net: "trainval.prototxt"
test_net: "test.prototxt"
test_iter: 200
# make test net, but don't invoke it from the solver itself
test_interval: 999999999
display: 20
average_loss: 20
lr_policy: "fixed"
# lr for unnormalized softmax
base_lr: 1e-10
# high momentum
momentum: 0.99
# no gradient accumulation
iter_size: 1
max_iter: 300000
weight_decay: 0.0005
snapshot: 5000
snapshot_prefix: "snapshot/train"
test_initialization: false
solve.py文件修改:
在这里郑重声明一下:如果训练fcn32s的网络模型,一定要修改solve.py,利用transplant的方式获取vgg16的网络权重.具体操作为:
import caffe
import surgery, score
import numpy as np
import os
import sys
try:
import setproctitle
setproctitle.setproctitle(os.path.basename(os.getcwd()))
except:
pass
vgg_weights = '../ilsvrc-nets/VGG_ILSVRC_16_layers.caffemodel'
vgg_proto = '../ilsvrc-nets/VGG_ILSVRC_16_layers_deploy.prototxt'
#weights = '../ilsvrc-nets/vgg16-fcn.caffemodel
# init
#caffe.set_device(int(sys.argv[1]))
caffe.set_device(0)
caffe.set_mode_gpu()
# solver = caffe.SGDSolver('solver.prototxt')
# solver.net.copy_from(weights)
solver = caffe.SGDSolver('solver.prototxt')
vgg_net = caffe.Net(vgg_proto, vgg_weights, caffe.TRAIN)
surgery.transplant(solver.net, vgg_net)
del vgg_net
# surgeries
interp_layers = [k for k in solver.net.params.keys() if 'up' in k]
surgery.interp(solver.net, interp_layers)
# scoring
test = np.loadtxt('../data/nyud/test.txt', dtype=str)
for _ in range(50):
solver.step(5000)
score.seg_tests(solver, False, test, layer='score')
可以看到我注释了:
#weights = '../ilsvrc-nets/vgg16-fcn.caffemodel
...
...
# solver = caffe.SGDSolver('solver.prototxt')
# solver.net.copy_from(weights)
添加了:
vgg_weights = '../ilsvrc-nets/VGG_ILSVRC_16_layers.caffemodel'
vgg_proto = '../ilsvrc-nets/VGG_ILSVRC_16_layers_deploy.prototxt'
...
...
solver = caffe.SGDSolver('solver.prototxt')
vgg_net = caffe.Net(vgg_proto, vgg_weights, caffe.TRAIN)
surgery.transplant(solver.net, vgg_net)
del vgg_net
关于transplant函