COCO数据集提取自己想要的类

前言

COCO数据集是一个很大的数据集，包括了语以分割、实例分割、目标检测等，因此它的标签就对应了几种不同的标注方式。最新的COCO2017对应了总共90个类别，但有时侯我们在做任务不需要这么多的类，只需要其中的一类或几类。本篇以提取“person”这一类的目标检测数据集为例，并将标签转化为我们通常使用的xml格式。

一、COCO数据集

COCO数据集官方下载，选择如下图所示进行下载，我需要的是目标检测标签，在“Train/Val annotations”中的“instances_ ”两个json文件。

二、安装COCO-PythonAPI(pycocotools)

1、Linux

pip install cython
git clone https://github.com/cocodataset/cocoapi.git
cd coco/PythonAPI
make

2、Windows

pip install cython
git clone https://github.com/cocodataset/cocoapi.git
cd coco/PythonAPI
python setup.py build_ext --inplace
python setup.py build_ext --install

3、关于Windows下pycocotools工具安装过程的bug
bug1
如下图所示，error: Microsoft Visual C++ 14.0 is required. Get it with "Build Tools for Visual Studio ”。这是因为没有安装“Visual C++”的编译环境，如果原本里已经配置了CUDA环境的，应该是可以正常编译的，因为安装CUDA的前提就是要安装“Visual Stdio C++”。

解决方案
网上有看多直接安装一个插件的解决方案，但不一定适用于所有人，暴力点直接安装“Visual Stdio C++”。百度云盘下载链接：https://pan.baidu.com/s/1iAWvBrRp8qjQokbA2w2yVQ 提取码：4dpw。

• Visual Stdio C++安装方式
  (1)解压压缩包，单击Visual软件，如下图所示，选择红色框内的选项，在单机右下角“安装”；
  
  (2)上步完成后，如下图所示，分为三个安装版本，看自己选择，选择其中一个安装即可，安装过程较慢，所需空间15G左右。
  

bug2

Visual Stdio编译环境安装后，再次执行安装指令后出现“failed with exit status 2”，如下图所示。

解决方案
修改setup.py文件,修改extra_compile_args为extra_compile_args=[’-std=c99’]即可。重新执行安装指令即可。

from setuptools import setup, Extension
import numpy as np

# To compile and install locally run "python setup.py build_ext --inplace"
# To install library to Python site-packages run "python setup.py build_ext install"

ext_modules = [
    Extension(
        'pycocotools._mask',
        sources=['../common/maskApi.c', 'pycocotools/_mask.pyx'],
        include_dirs = [np.get_include(), '../common'],
        #extra_compile_args=['-Wno-cpp', '-Wno-unused-function', '-std=c99'],
        extra_compile_args=['-std=c99'],
    )
]

setup(
    name='pycocotools',
    packages=['pycocotools'],
    package_dir = {'pycocotools': 'pycocotools'},
    install_requires=[
        'setuptools>=18.0',
        'cython>=0.27.3',
        'matplotlib>=2.1.0'
    ],
    version='2.0',
    ext_modules= ext_modules
)

三、类提取+Json转xml

上面都是pycocotools的安装，使用pip list查看是否已经安装，安装成功后执行以下代码。但我们还需要修改几个参数来是适配自己的需要，具体如下：

• savepath：自己需要提取的类别图片和标签的保存位置，提前创建好“images”和“Annotations”文件夹
• img_dir = savepath + ‘images/’：类别的图片保存地址
• anno_dir = savepath + ‘Annotations/’：类别的标签保存地址
• classes_names：自己需要的类别
• dataDir：COCO数据集的地址

from pycocotools.coco import COCO
import os
import shutil
from tqdm import tqdm
import skimage.io as io
import matplotlib.pyplot as plt
import cv2
from PIL import Image, ImageDraw

# the path you want to save your results for coco to voc
savepath = "E:/coco2017/result/"
img_dir = savepath + 'images/'
anno_dir = savepath + 'Annotations/'
# datasets_list=['train2014', 'val2014']
datasets_list = ['train2017','val2017']

#classes_names = ['car', 'bicycle', 'person', 'motorcycle', 'bus', 'truck']
classes_names = ['person']
# Store annotations and train2014/val2014/... in this folder
dataDir = 'E:/coco2017/annotations_trainval2017'

headstr = """\
<annotation>
    <folder>VOC</folder>
    <filename>%s</filename>
    <source>
        <database>My Database</database>
        <annotation>COCO</annotation>
        <image>flickr</image>
        <flickrid>NULL</flickrid>
    </source>
    <owner>
        <flickrid>NULL</flickrid>
        <name>company</name>
    </owner>
    <size>
        <width>%d</width>
        <height>%d</height>
        <depth>%d</depth>
    </size>
    <segmented>0</segmented>
"""
objstr = """\
    <object>
        <name>%s</name>
        <pose>Unspecified</pose>
        <truncated>0</truncated>
        <difficult>0</difficult>
        <bndbox>
            <xmin>%d</xmin>
            <ymin>%d</ymin>
            <xmax>%d</xmax>
            <ymax>%d</ymax>
        </bndbox>
    </object>
"""

tailstr = '''\
</annotation>
'''


# if the dir is not exists,make it,else delete it
def mkr(path):
    if os.path.exists(path):
        shutil.rmtree(path)
        os.mkdir(path)
    else:
        os.mkdir(path)


mkr(img_dir)
mkr(anno_dir)


def id2name(coco):
    classes = dict()
    for cls in coco.dataset['categories']:
        classes[cls['id']] = cls['name']
    return classes


def write_xml(anno_path, head, objs, tail):
    f = open(anno_path, "w")
    f.write(head)
    for obj in objs:
        f.write(objstr % (obj[0], obj[1], obj[2], obj[3], obj[4]))
    f.write(tail)


def save_annotations_and_imgs(coco, dataset, filename, objs):
    # eg:COCO_train2014_000000196610.jpg-->COCO_train2014_000000196610.xml
    anno_path = anno_dir + filename[:-3] + 'xml'
    img_path = dataDir + '/'+dataset + '/' + filename
    print("img_path:",img_path)
    dst_imgpath = img_dir + filename

    img = cv2.imread(img_path)
    if (img.shape[2] == 1):
        print(filename + " not a RGB image")
        return
    shutil.copy(img_path, dst_imgpath)

    head = headstr % (filename, img.shape[1], img.shape[0], img.shape[2])
    tail = tailstr
    write_xml(anno_path, head, objs, tail)


def showimg(coco, dataset, img, classes, cls_id, show=True):
    global dataDir
    I = Image.open('%s/%s/%s' % (dataDir, dataset, img['file_name']))
    # 通过id，得到注释的信息
    annIds = coco.getAnnIds(imgIds=img['id'], catIds=cls_id, iscrowd=None)
    # print(annIds)
    anns = coco.loadAnns(annIds)
    # print(anns)
    # coco.showAnns(anns)
    objs = []
    for ann in anns:
        class_name = classes[ann['category_id']]
        if class_name in classes_names:
            print(class_name)
            if 'bbox' in ann:
                bbox = ann['bbox']
                xmin = int(bbox[0])
                ymin = int(bbox[1])
                xmax = int(bbox[2] + bbox[0])
                ymax = int(bbox[3] + bbox[1])
                obj = [class_name, xmin, ymin, xmax, ymax]
                objs.append(obj)
                draw = ImageDraw.Draw(I)
                draw.rectangle([xmin, ymin, xmax, ymax])
    if show:
        plt.figure()
        plt.axis('off')
        plt.imshow(I)
        plt.show()

    return objs


for dataset in datasets_list:
    # ./COCO/annotations/instances_train2014.json
    annFile = '{}/annotations/instances_{}.json'.format(dataDir, dataset)

    # COCO API for initializing annotated data
    coco = COCO(annFile)
    '''
    COCO 对象创建完毕后会输出如下信息:
    loading annotations into memory...
    Done (t=0.81s)
    creating index...
    index created!
    至此, json 脚本解析完毕, 并且将图片和对应的标注数据关联起来.
    '''
    # show all classes in coco
    classes = id2name(coco)
    print(classes)
    # [1, 2, 3, 4, 6, 8]
    classes_ids = coco.getCatIds(catNms=classes_names)
    print(classes_ids)
    for cls in classes_names:
        # Get ID number of this class
        cls_id = coco.getCatIds(catNms=[cls])
        img_ids = coco.getImgIds(catIds=cls_id)
        print(cls, len(img_ids))
        # imgIds=img_ids[0:10]
        for imgId in tqdm(img_ids):
            img = coco.loadImgs(imgId)[0]
            filename = img['file_name']
            print("filename:",filename)
            print("dataset:",dataset)
            objs = showimg(coco, dataset, img, classes, classes_ids, show=False)
            print(objs)
            save_annotations_and_imgs(coco, dataset, filename, objs)

执行上述代码后，运行时间较长，以我提取的目标检测的一个类别为例，需要大致5小时。整个程序包可通过百度云盘下载：https://pan.baidu.com/s/1xPXvNYSjKroxO3Sb0BgsWg 提取码：zual

四、参考

[1] 博客