应用在很多目标检测框架中,常常需要coco数据集的形式,但我们通常只有voc数据格式
本篇文章主要使用python实现voc数据集转化为coco数据集
目录
1 voc2012数据集了解
其中Annotations存放的是五个文件夹,但针对目标检测而言,只有前面三个文件夹存放的内容是有意义的,文件目录如下
voc
|--------Annotations
| |___xxx.xml
| |___xxx.xml
...
|--------ImageSets
| |___Main
| |_____test.txt
| |_____train.txt
| |_____trainval.txt
| |_____val.txt
|--------JPEGImages
| |___xxx.jpg
| |___xxx.jpg
...
2 coco数据集的了解
不同voc,一张图片对应一个xml文件,coco是直接将所有的图片信息及其对应的bbox信息写在了同一个json下
首先需要了解一下coco的基本结构
coco
|______annotations # 存放标注信息
| |__train.json
| |__val.json
| |__test.json
|______trainset # 存放训练集图像
|______valset # 存放验证集图像
|______testset # 存放测试集图像
一个json包含如下信息
{
"info" : info,
"licenses" : [license],
"images" : [image],
"annotations" : [annataton],
"categories" : [category]
}
通过上面的json整体结构可以看出,info这个key对应的值的类型是一个字典;licenses、images、annotations和categories这四个key对应的值的类型都是一个列表,列表当中存储的数据类型依旧是字典。
我们可以通过len(List)的方式得到images、annotations、categories这三个列表的长度,也就得到了以下内容。
=======================================
(1)images字段列表元素的长度 = 划入训练集(或者测试集)的图片的数量;
(2)annotations字段列表元素的数量 = 训练集(或者测试集)中bounding box的数量;
(3)categories字段列表元素的数量 = 类别的数量
=======================================
3 代码流程
3.1 首先是获取图片的名称生成txt文件
运行 get_txt.py文件
import os
saveBasePath='D:/Python_File' #存放txt文件的地址
jpgfilepath='D:/python_data/Voc_to_coco/Coco/test2017' #jpg图片的地址
ftest = open(os.path.join(saveBasePath, 'test.txt'), 'w') #生成的txt文件
temp_jpg = os.listdir(jpgfilepath) #获得一个列表
total_jpg = []
print(temp_jpg)
for jpg in temp_jpg:
if jpg.endswith(".jpg"):
total_jpg.append(jpg)
num=len(total_jpg)
list=range(num)
for i in list:
name = total_jpg[i][:-4] + '\n' #只取.jpg前面的数字,并且存进name
ftest.write(name)
ftest.close()
生产的test.txt内容
3.2 运行下面的文件
运行voc_to_coco.py文件
import sys
import os
import json
import xml.etree.ElementTree as ET
START_BOUNDING_BOX_ID = 1
PRE_DEFINE_CATEGORIES = {"defect":1,"insulator":2} #修改的地方,修改为自己的类别
# If necessary, pre-define category and its id
# PRE_DEFINE_CATEGORIES = {"aeroplane": 1, "bicycle": 2, "bird": 3, "boat": 4,
# "bottle":5, "bus": 6, "car": 7, "cat": 8, "chair": 9,
# "cow": 10, "diningtable": 11, "dog": 12, "horse": 13,
# "motorbike": 14, "person": 15, "pottedplant": 16,
# "sheep": 17, "sofa": 18, "train": 19, "tvmonitor": 20}
def get(root, name):
vars = root.findall(name)
return vars
def get_and_check(root, name, length):
vars = root.findall(name)
if len(vars) == 0:
raise NotImplementedError('Can not find %s in %s.'%(name, root.tag))
if length > 0 and len(vars) != length:
raise NotImplementedError('The size of %s is supposed to be %d, but is %d.'%(name, length, len(vars)))
if length == 1:
vars = vars[0]
return vars
def get_filename_as_int(filename):
try:
filename = os.path.splitext(filename)[0]
return int(filename)
except:
raise NotImplementedError('Filename %s is supposed to be an integer.'%(filename))
#xml_list为xml文件存放的txt文件名 xml_dir为真实xml的存放路径 json_file为存放的json路径
def convert(xml_list, xml_dir, json_file):
list_fp = open(xml_list, 'r')
json_dict = {"images":[], "type": "instances", "annotations": [],
"categories": []}
categories = PRE_DEFINE_CATEGORIES
bnd_id = START_BOUNDING_BOX_ID
for line in list_fp:
line = line.strip()
line=line+".xml"
print("Processing %s"%(line))
xml_f = os.path.join(xml_dir, line)
tree = ET.parse(xml_f)
root = tree.getroot()
path = get(root, 'path')
if len(path) == 1:
filename = os.path.basename(path[0].text)
elif len(path) == 0:
filename = get_and_check(root, 'filename', 1).text
else:
raise NotImplementedError('%d paths found in %s'%(len(path), line))
## The filename must be a number
image_id = get_filename_as_int(filename)
size = get_and_check(root, 'size', 1)
width = int(get_and_check(size, 'width', 1).text)
height = int(get_and_check(size, 'height', 1).text)
image = {'file_name': filename, 'height': height, 'width': width,
'id':image_id}
json_dict['images'].append(image)
## Cruuently we do not support segmentation
# segmented = get_and_check(root, 'segmented', 1).text
# assert segmented == '0'
for obj in get(root, 'object'):
category = get_and_check(obj, 'name', 1).text
if category not in categories:
new_id = len(categories)
categories[category] = new_id
category_id = categories[category]
bndbox = get_and_check(obj, 'bndbox', 1)
xmin = int(get_and_check(bndbox, 'xmin', 1).text) - 1
ymin = int(get_and_check(bndbox, 'ymin', 1).text) - 1
xmax = int(get_and_check(bndbox, 'xmax', 1).text)
ymax = int(get_and_check(bndbox, 'ymax', 1).text)
assert(xmax > xmin)
assert(ymax > ymin)
o_width = abs(xmax - xmin)
o_height = abs(ymax - ymin)
ann = {'area': o_width*o_height, 'iscrowd': 0, 'image_id':
image_id, 'bbox':[xmin, ymin, o_width, o_height],
'category_id': category_id, 'id': bnd_id, 'ignore': 0,
'segmentation': []}
json_dict['annotations'].append(ann)
bnd_id = bnd_id + 1
for cate, cid in categories.items():
cat = {'supercategory': 'none', 'id': cid, 'name': cate}
json_dict['categories'].append(cat)
json_fp = open(json_file, 'w')
json_str = json.dumps(json_dict)
json_fp.write(json_str)
json_fp.close()
list_fp.close()
if __name__ == '__main__':
#xml_list为xml文件存放的txt文件名 xml_dir为真实xml的存放路径 json_file为存放的json路径
xml_list='D:/python_data/Voc_to_coco/VOC2007/ImageSets/Main/test.txt'
xml_dir='D:/python_data/Voc_to_coco/VOC2007/Annotations/'
json_dir='D:/python_data/Voc_to_coco/Coco/annotations/test.json' #注意!!!这里test.json先要自己创建,不然 #程序回报权限不足
convert(xml_list, xml_dir, json_dir)
生成一个json文件