caffe学习（11）：使用迁移学习预训练模型训练

数据集还是用GTSDB，我是打算用之前VOC0712数据集针对普通目标20类的模型，迁移到检测GTSDB上来。实验发现用迁移学习还是能提升一定精度的。
预训练模型：VGG_VOC0712_SSD_300x300_iter_120000.caffemodel这个是我自己训练的。

目标

数据集GTSDB
44小类目标检测并分类

数据集处理

参照caffe学习（10）：交通标志目标检测训练整体流程数据集处理部分。还是原来的图片，格式已经转换，不需要再运行ppm2jpg.py。
我这边是把真值中的0-42xml中标记为1-43，背景标记作0。gt2xml.py代码修改如下：

#!/usr/bin/env python
#-*- coding:utf-8 -*-
import sys
import os
import codecs
import cv2
reload(sys)
sys.setdefaultencoding('utf8')

root = r'/home/jqy/data/gtsdbjpg/all/xml/' # output xml path
fp = open('gt.txt') # path of gt.txt
#fp2 = open('train.txt', 'w') # path of train.txt
uavinfo = fp.readlines()

def get_label(label):
    return label+1
    
for i in range(len(uavinfo)):
    line = uavinfo[i]
    line = line.strip().split(';') 
    line[0] = "/home/jqy/data/gtsdbjpg/all/"+str(line[0]) # need to write image path
    img = cv2.imread(line[0])
    print line[0]
    sp = img.shape
    height = sp[0]
    width = sp[1]
    depth = sp[2]
    info1 = line[0].split('/')[-1]
    info2 = info1.split('.')[0]

    l_pos1 = line[1]
    l_pos2 = line[2]
    r_pos1 = line[3]
    r_pos2 = line[4]
    name = int(line[5])
    lable = get_label(name)
    if(os.path.exists(root + info2 + '.xml') == False):
        #fp2.writelines(info2 + '\n')
        with codecs.open(root + info2 + '.xml', 'w', 'utf-8') as xml:
            # xml.write('<?xml version="1.0" encoding="UTF-8"?>\n')
            xml.write('<annotation>\n')
            xml.write('\t<folder>' + 'GTSDB' + '</folder>\n')
            xml.write('\t<filename>' + info1 + '</filename>\n')
            xml.write('\t<source>\n')
            xml.write('\t\t<database>The GTSDB Database</database>\n')
            xml.write('\t</source>\n')
            xml.write('\t<size>\n')
            xml.write('\t\t<width>'+ str(width) + '</width>\n')
            xml.write('\t\t<height>'+ str(height) + '</height>\n')
            xml.write('\t\t<depth>' + str(depth) + '</depth>\n')
            xml.write('\t</size>\n')
            xml.write('\t\t<segmented>0</segmented>\n')
            xml.write('\t<object>\n')
            xml.write('\t\t<name>' + str(lable) + '</name>\n')
            xml.write('\t\t<pose>Unspecified</pose>\n')
            xml.write('\t\t<truncated>0</truncated>\n')
            xml.write('\t\t<difficult>0</difficult>\n')
            xml.write('\t\t<bndbox>\n')
            xml.write('\t\t\t<xmin>' + l_pos1 + '</xmin>\n')
            xml.write('\t\t\t<ymin>' + l_pos2 + '</ymin>\n')
            xml.write('\t\t\t<xmax>' + r_pos1 + '</xmax>\n')
            xml.write('\t\t\t<ymax>' + r_pos2 + '</ymax>\n')
            xml.write('\t\t</bndbox>\n')
            xml.write('\t</object>\n')
            xml.write('</annotation>')
    else:
        with codecs.open(root + info2 + '.xml', 'r', 'utf-8') as xml:
            lines = xml.readlines()
        with codecs.open(root + info2 + '.xml', 'w', 'utf-8') as xml:
            for line in lines:
                if "</annotation>" in line:
                    continue
                xml.write(line)
            xml.write('\t<object>\n')
            xml.write('\t\t<name>' + str(lable) + '</name>\n')
            xml.write('\t\t<pose>Unspecified</pose>\n')
            xml.write('\t\t<truncated>0</truncated>\n')
            xml.write('\t\t<difficult>0</difficult>\n')
            xml.write('\t\t<bndbox>\n')
            xml.write('\t\t\t<xmin>' + l_pos1 + '</xmin>\n')
            xml.write('\t\t\t<ymin>' + l_pos2 + '</ymin>\n')
            xml.write('\t\t\t<xmax>' + r_pos1 + '</xmax>\n')
            xml.write('\t\t\t<ymax>' + r_pos2 + '</ymax>\n')
            xml.write('\t\t</bndbox>\n')
            xml.write('\t</object>\n')
            xml.write('</annotation>')
for i in range(900):
    name = str(i).zfill(5)
    img_path = "/home/jqy/data/gtsdbjpg/all/" + name + '.jpg'
    print(img_path)
    img = cv2.imread(str(img_path))
    sp = img.shape
    height = sp[0]
    width = sp[1]
    depth = sp[2]
    info1 = img_path.split('/')[-1]
    info2 = info1.split('.')[0]
    if(os.path.exists(root + info2 + '.xml') == False):
        #fp2.writelines(info2 + '\n')
        with codecs.open(root + info2 + '.xml', 'w', 'utf-8') as xml:
            # xml.write('<?xml version="1.0" encoding="UTF-8"?>\n')
            xml.write('<annotation>\n')
            xml.write('\t<folder>' + 'GTSDB' + '</folder>\n')
            xml.write('\t<filename>' + info1 + '</filename>\n')
            xml.write('\t<source>\n')
            xml.write('\t\t<database>The GTSDB Database</database>\n')
            xml.write('\t</source>\n')
            xml.write('\t<size>\n')
            xml.write('\t\t<width>'+ str(width) + '</width>\n')
            xml.write('\t\t<height>'+ str(height) + '</height>\n')
            xml.write('\t\t<depth>' + str(depth) + '</depth>\n')
            xml.write('\t</size>\n')
            xml.write('\t\t<segmented>0</segmented>\n')
            xml.write('</annotation>')
#fp2.close()

运行。

训练前数据处理

参照caffe学习（10）：交通标志目标检测训练整体流程训练前数据处理部分。训练集测试集划分还是照原来，不需要运行train_test_separation.py。create_list.sh也不需要运行。
labelmap_voc.prototxt需要修改：背景标记为0。

item {
  name: "0"
  label: 0
  display_name: "none"
}
item {
  name: "1"
  label: 1
  display_name: "speed limit 20 (prohibitory)"
}
item {
  name: "2"
  label: 2
  display_name: "speed limit 30 (prohibitory)"
}
item {
  name: "3"
  label: 3
  display_name: "speed limit 50 (prohibitory)"
}
item {
  name: "4"
  label: 4
  display_name: "speed limit 60 (prohibitory)"
}
item {
  name: "5"
  label: 5
  display_name: "speed limit 70 (prohibitory)"
}
item {
  name: "6"
  label: 6
  display_name: "speed limit 80 (prohibitory)"
}
item {
  name: "7"
  label: 7
  display_name: "restriction ends 80 (other)"
}
item {
  name: "8"
  label: 8
  display_name: "speed limit 100 (prohibitory)"
}
item {
  name: "9"
  label: 9
  display_name: "speed limit 120 (prohibitory)"
}
item {
  name: "10"
  label: 10
  display_name: "no overtaking (prohibitory)"
}
item {
  name: "11"
  label: 11
  display_name: "no overtaking (trucks) (prohibitory)"
}
item {
  name: "12"
  label: 12
  display_name: "priority at next intersection (danger)"
}
item {
  name: "13"
  label: 13
  display_name: "priority road (other)"
}
item {
  name: "14"
  label: 14
  display_name: "give way (other)"
}
item {
  name: "15"
  label: 15
  display_name: "stop (other)"
}
item {
  name: "16"
  label: 16
  display_name: "no traffic both ways (prohibitory)"
}
item {
  name: "17"
  label: 17
  display_name: "no trucks (prohibitory)"
}
item {
  name: "18"
  label: 18
  display_name: "no entry (other)"
}
item {
  name: "19"
  label: 19
  display_name: "danger (danger)"
}
item {
  name: "20"
  label: 20
  display_name: "bend left (danger)"
}
item {
  name: "21"
  label: 21
  display_name: "bend right (danger)"
}
item {
  name: "22"
  label: 22
  display_name: "bend (danger)"
}
item {
  name: "23"
  label: 23
  display_name: "bend (danger)"
}
item {
  name: "24"
  label: 24
  display_name: "slippery road (danger)"
}
item {
  name: "25"
  label: 25
  display_name: "road narrows (danger)"
}
item {
  name: "26"
  label: 26
  display_name: "construction (danger)"
}
item {
  name: "27"
  label: 27
  display_name: "traffic signal (danger)"
}
item {
  name: "28"
  label: 28
  display_name: "pedestrian crossing (danger)"
}
item {
  name: "29"
  label: 29
  display_name: "school crossing (danger)"
}
item {
  name: "30"
  label: 30
  display_name: "cycles crossing (danger)"
}
item {
  name: "31"
  label: 31
  display_name: "snow (danger)"
}
item {
  name: "32"
  label: 32
  display_name: "animals (danger)"
}
item {
  name: "33"
  label: 33
  display_name: "restriction ends (other)"
}
item {
  name: "34"
  label: 34
  display_name: "go right (mandatory)"
}
item {
  name: "35"
  label: 35
  display_name: "go left (mandatory)"
}
item {
  name: "36"
  label: 36
  display_name: "go straight (mandatory)"
}
item {
  name: "37"
  label: 37
  display_name: "go right or straight (mandatory)"
}
item {
  name: "38"
  label: 38
  display_name: "go left or straight (mandatory)"
}
item {
  name: "39"
  label: 39
  display_name: "keep right (mandatory)"
}
item {
  name: "40"
  label: 40
  display_name: "keep left (mandatory)"
}
item {
  name: "41"
  label: 41
  display_name: "roundabout (mandatory)"
}
item {
  name: "42"
  label: 42
  display_name: "restriction ends (overtaking) (other)"
}
item {
  name: "43"
  label: 43
  display_name: "restriction ends (overtaking (trucks)) (other)"
}

运行create_data.sh创建44类对应的lmdb文件。

训练

修改ssd_gtsdb.py

基于caffe学习（10）：交通标志目标检测训练整体流程中ssd_gtsdb.py。作如下改动：


运行ssd_gtsdb.py。
这边生成的train.prototxt，test.prototxt，deploy.prototxt中的多尺度检测的卷积层（conv4_3_norm_mbox_conf、fc7_mbox_conf、conv6_2_mbox_conf、conv7_2_mbox_conf、conv8_2_mbox_conf、conv9_2_mbox_conf）的num_output参数会随着num_classes改变，如何改变根据SSD论文原理。

模型迁移

模型能够迁移是因为最后训练出来的模型其实只记载了conv层的权值参数。最后不管是train.prototxt中的MultiBoxLoss层还是test.prototxt中的DetectionOutput层、DetectionEvaluate层。都是自定义了层，来使用.caffemodel中记录的神经网络的权值参数。大概是通过layer的名字去训练参数的，所以名字不同就会训练心的参数。所以以下操作，就是conf相关参数都是重新训练的。

网络结构文件.prototxt修改

train.prototxt，test.prototxt，deploy.prototxt（这三个文件生成在models/VGGNet/gtsdb_voc/SSD_300x300/下，主要是solver.prototxt中调取了这个路径，当然solver.prototxt路径也可以手动改）手动修改所有与conf相关的层的名字。我的操作就是conf全部替换成conf1。

注意train.prototxt中也改变了，改回conf。
test.prototxt中也改变了，改回conf。
deploy.prototxt中也改变了，改回conf。

.sh修改

修改VGG_gtsdb_voc_SSD_300x300.sh为如下：

cd /home/jqy/jqy_caffe/caffe-gpu/caffe-ssd
./build/tools/caffe train \
--solver="models/VGGNet/gtsdb_voc/SSD_300x300/solver.prototxt" \
--weights="jobs/VGGNet/gtsdb_voc/SSD_300x300/VGG_VOC0712_SSD_300x300_iter_120000.caffemodel" \
--gpu 0,1 2>&1 | tee jobs/VGGNet/gtsdb_voc/SSD_300x300/VGG_gtsdb_voc_SSD_300x300.log

运行VGG_gtsdb_voc_SSD_300x300.sh