一、训练自己标注的VOC数据集
使用Faster-ILOD训练自己标注的数据集,数据集已经转换为VOC格式
之前已经可以顺利训练VOC2007数据集,可参考
Faster-ILOD、maskrcnn_benchmark安装过程及遇到问题
主要修改以下文件:
1.maskrcnn_benchmark/config/paths_catalog.py
对应路径修改为自己数据集所在的路径
"voc_2007_train": {
"data_dir": "/data/taxi_data/VOC2007_all",
"split": "train"
},
"voc_2007_val": {
"data_dir": "/data/taxi_data/VOC2007_all",
"split": "val"
},
"voc_2007_test": {
"data_dir": "/data/taxi_data/VOC2007_all",
"split": "test"
},
2.maskrcnn_benchmark/data/voc.py
修改为数据集中包含类别 我主要进行出租车和私家车的识别
# CLASSES = ("__background__ ", "aeroplane", "bicycle", "bird", "boat", "bottle", "bus", "car", "cat", "chair", "cow",
# "diningtable", "dog", "horse", "motorbike", "person", "pottedplant", "sheep", "sofa", "train", "tvmonitor")
CLASSES = ("__background__ ", "Taxi","Other Car")
3.修改configs/e2e_faster_rcnn_R_50_C4_1x.yaml
修改类别数量等,我一次把两个类都训练了。
NUM_CLASSES: 3 # total classes
NAME_OLD_CLASSES: []
#NAME_NEW_CLASSES: ["aeroplane", "bicycle", "bird", "boat", "bottle", "bus", "car", "cat", "chair", "cow", "diningtable", "dog",
# "horse", "motorbike", "person","pottedplant", "sheep", "sofa", "train","tvmonitor"]
# NAME_EXCLUDED_CLASSES: [ ]
NAME_NEW_CLASSES: ["Taxi","Other Car"]
NAME_EXCLUDED_CLASSES: []
4.运行
python tools/train_first_step.py --config-file="./configs/e2e_faster_rcnn_R_50_C4_1x.yaml"
二、遇到问题
1.没有对应的Taxi_train.txt 等文件
因为是做增量学习,会针对每一个类别进行处理,原始下载的voc文件中是包含每一类别对应的train、test等txt文件,但对自己标注数据集转换为voc格式的数据集时中转换为train.txt,test.txt,trainval.txt以及val.txt。没有对应的txt文件,因此要生成对应的文件。
观察元数据集中每一类别对应的txt文件,每一行对应两列,一列为图片名称,另一列是一位数字,应该值为1.-1.0.
-1代表该类不存在此图片中
1位存在此图片中
0为困难样本
(个人觉得是这样,不对的欢迎指正)
根据xml文件和train.txt,test.txt,val.txt进行对应类别的转换。代码如下:
我生成对应文件时只将存在此类别的图片名称写入,在数据读取的时候报错就改了一下代码,可以修改一下代码,与原数据集保持一致。
names = locals()
dir_path = "D:/achenf/data/jiaotong_data/1taxi_train_data/VOC/VOC2007/"
# 读取train.txt/test.txt/val.txt文件
f = open("D:/achenf/data/jiaotong_data/1taxi_train_data/VOC/VOC2007/ImageSets/Main/train.txt", 'r')
# 包含类别
classes = ['Taxi','Other Car']
# 写入文档名称
txt_path = dir_path+"by_classes/"+"Other Car_train.txt"
fp_w = open(txt_path, 'w')
for line in f.readlines():
if not line:
break
n= line[:-1]
xmlpath=dir_path+"Annotations/"+n+'.xml'
fp = open(xmlpath)
xmllines = fp.readlines()
ind_start = []
ind_end = []
lines_id_start = xmllines[:]
lines_id_end = xmllines[:]
# 修改对应类别Other Car/Taxi
classes1 = ' <name>Other Car</name>\n'
while " <object>\n" in lines_id_start:
a = lines_id_start.index(" <object>\n")
ind_start.append(a)
lines_id_start[a] = "delete"
while " </object>\n" in lines_id_end:
b = lines_id_end.index(" </object>\n")
ind_end.append(b)
lines_id_end[b] = "delete"
# names中存放所有的object块
i = 0
for k in range(0, len(ind_start)):
names['block%d' % k] = []
for j in range(0, len(classes)):
if classes[j] in xmllines[ind_start[i] + 1]:
a = ind_start[i]
for o in range(ind_end[i] - ind_start[i] + 1):
names['block%d' % k].append(xmllines[a + o])
break
i += 1
# 在给定的类中搜索,若存在则,写入object块信息
a = 0
if len(ind_start)>0:
# xml头
string_start = xmllines[0:ind_start[0]]
# xml尾
string_end = [xmllines[len(xmllines) - 1]]
flag = False
for k in range(0, len(ind_start)):
if classes1 in names['block%d' % k]:
flag = True
a += 1
string_start += names['block%d' % k]
string_start += string_end
# 如果存在写入
if flag:
fp_w.write(n+" "+"1"+'\n')
fp.close()
fp_w.close()
得到以下文件:
2.x[2] == ‘0’ 超出下标
对每一行就行分割时,如果值为1,源数据集中txt文件分解为[‘000000’,‘’,‘1’],我生成的文件分解为[‘000000’,‘1’],因为没有困难样本,直接将这几行注释
for i in range(len(buff)):
x = buff[i]
x = x.split(' ')
if x[1] == '-1':
pass
# elif x[2] == '0': # include difficult level object
# if self.is_train:
# pass
# else:
# img_ids_per_category.append(x[0])
# self.ids.append(x[0])
else:
img_ids_per_category.append(x[0])
self.ids.append(x[0])
以及本人的xml文件中也不存在difficult的标签,所以遇到此行代码difficult = int(obj.find("difficult").text) == 1
也会报错,所以直接将difficult赋值为False
for obj in target.iter("object"):
difficult = False
# difficult = int(obj.find("difficult").text) == 1
if not self.keep_difficult and difficult:
continue
3.ValueError: invalid literal for int() with base 10: ‘0.0’
报错出在此行bndbox = tuple(map(lambda x: x - TO_REMOVE, list(map(int, box))))
,应该是str直接转为int的问题,于是我们先将其转换为float,在转换为int,修改如下:bndbox = tuple(map(lambda x: x - TO_REMOVE, list(map(int, map(float,box)))))
就ok了
学习到一个新知识点,map()函数可以实现批量数据类型转换。
4.Pytorch报错 CUDA error: device-side assert triggered
参考:https://blog.csdn.net/veritasalice/article/details/111917185