本文是结论,详细过程参考 https://blog.csdn.net/yinglang19941010/article/details/102790168
问题描述
使用Pascal的coco格式标注文件是Detectron代码提供的,下载地址为https://github.com/facebookresearch/Detectron/blob/master/detectron/datasets/data/README.md#coco-minival-annotations
使用coco评测标准测出的AP50和voc评测标准测出的AP50不一致,相差好几个点(4~5).
问题解决
修改评测代码maskrcnn_benchmark/data/datasets/evaluation/coco/cocoeval.py _prepare函数
# orgin code
# gt['ignore'] = 'iscrowd' in gt and gt['iscrowd']
# changed code
gt['ignore'] = gt['ignore'] if 'ignore' in gt else 0
gt['ignore'] = ('iscrowd' in gt and gt['iscrowd']) or gt['ignore'] # changed by hui
问题简单分析
Pascal的coco格式标注文件是Detectron代码提供的,下载地址为https://github.com/facebookresearch/Detectron/blob/master/detectron/datasets/data/README.md#coco-minival-annotations
加载查看annotations字段,发现他有一个ignore字段,估计应该是pascal里的difficult字段之类的
In [2]: import json
In [3]: jd_gt = json.load(open('pascal_test2007.json'))
In [4]: jd_gt['annotations'][0]
Out[4]:
{'segmentation': [[47, 239, 47, 371, 195, 371, 195, 239]],
'area': 19536,
'iscrowd': 0,
'image_id': 1,
'bbox': [47, 239, 148, 132],
'category_id': 12,
'id': 1,
'ignore': 0}
但是标准的coco评测代码里并没有ignroe字段,因此,即使ignore不为0,也不会被处理,但是Pascal VOC中正好有ignore不为0的数据.
print(len(jd_gt['annotations']))
ann_gt1 = [a for a in jd_gt['annotations'] if a['iscrowd']==0]
print(len(ann_gt1))
ann_gt2 = [a for a in jd_gt['annotations'] if a['ignore']==0]
print(len(ann_gt2))
Out[]:
14976
14976
12032
因此会有这个问题,即使使用ground-truth作为检测结果AP也只有80%