随笔--使用pycocoevalcap评估生成文本质量

官方示例代码：

from pycocotools.coco import COCO
from pycocoevalcap.eval import COCOEvalCap

annotation_file = 'captions_val2014.json'
results_file = 'captions_val2014_fakecap_results.json'

# create coco object and coco_result object
coco = COCO(annotation_file)
coco_result = coco.loadRes(results_file)

# create coco_eval object by taking coco and coco_result
coco_eval = COCOEvalCap(coco, coco_result)

# evaluate on a subset of images by setting
# coco_eval.params['image_id'] = coco_result.getImgIds()
# please remove this line when evaluating the full validation set
coco_eval.params['image_id'] = coco_result.getImgIds()

# evaluate results
# SPICE will take a few minutes the first time, but speeds up due to caching
coco_eval.evaluate()

# print output evaluation scores
for metric, score in coco_eval.eval.items():
    print(f'{metric}: {score:.3f}')

 重点一：

annotation_file用于存放ground truth

annotation_file文件内部格式可参考COCO数据集的标注格式 - 知乎

重点二：

results_file即用于存放模型预测结果的文件

results_file的格式

[{"image_id": 404464, "caption": "black and white photo of a man standing in front of a building"},......]

https://github.com/salaniz/pycocoevalcap/blob/master/example/coco_eval_example.py