使用pycocotools打印各个类别的AP值及IOU=0.5时的APS、APM及APL

关于pycocotools

pycocotools是用于处理 COCO 数据集的 Python 工具包，提供数据加载、评估工具和可视化功能。它简化了目标检测和图像分割任务，比如计算检测精度、召回率和 IOU（Intersection over Union）。

安装pycocotools

pip install pycocotools

获取各个类别的AP0.5值

首先需要找到pycocotools包所在的位置，其目录结构如下：
 

修改cocoeval.py

summarize方法用于计算平均精度（AP）、平均召回（AR）等重要性能指标。

新增summarize_2方法，用于计算特定类别的AP值，classId不能为None，summarize_2方法可以放在summarize方法下面。

    def summarize_2(self, classId=None):
        # classId不能为None
        assert classId is not None
        def _summarize(ap=1, iouThr=None, areaRng='all', maxDets=100):
            p = self.params
            iStr = ' {:<18} {} @[ IoU={:<9} | area={:>6s} | maxDets={:>3d} ] = {:0.3f}'
            titleStr = 'Average Precision' if ap == 1 else 'Average Recall'
            typeStr = '(AP)' if ap == 1 else '(AR)'
            iouStr = '{:0.2f}:{:0.2f}'.format(p.iouThrs[0], p.iouThrs[-1]) \
                if iouThr is None else '{:0.2f}'.format(iouThr)

            aind = [i for i, aRng in enumerate(p.areaRngLbl) if aRng == areaRng]
            mind = [i for i, mDet in enumerate(p.maxDets) if mDet == maxDets]

            if ap == 1:
                # dimension of precision: [TxRxKxAxM]
                s = self.eval['precision']
                # IoU
                if iouThr is not None:
                    t = np.where(iouThr == p.iouThrs)[0]
                    s = s[t]

                # 计算某一类别指标
                s = s[:, :, classId, aind, mind]

            else:
                # dimension of recall: [TxKxAxM]
                s = self.eval['recall']
                if iouThr is not None:
                    t = np.where(iouThr == p.iouThrs)[0]
                    s = s[t]
                # 计算某一类别指标
                s = s[:, classId, aind, mind]

            if len(s[s > -1]) == 0:
                mean_s = -1
            else:
                mean_s = np.mean(s[s > -1])

            print_string = iStr.format(titleStr, typeStr, iouStr, areaRng, maxDets, mean_s)
            return mean_s, print_string

        stats, print_list = [0] * 15, [""] * 12
        stats[0], print_list[0] = _summarize(1)
        stats[1], print_list[1] = _summarize(1, iouThr=.5, maxDets=self.params.maxDets[2])
        stats[2], print_list[2] = _summarize(1, iouThr=.75, maxDets=self.params.maxDets[2])
        stats[3], print_list[3] = _summarize(1, areaRng='small', maxDets=self.params.maxDets[2])
        stats[4], print_list[4] = _summarize(1, areaRng='medium', maxDets=self.params.maxDets[2])
        stats[5], print_list[5] = _summarize(1, areaRng='large', maxDets=self.params.maxDets[2])
        stats[6], print_list[6] = _summarize(0, maxDets=self.params.maxDets[0])
        stats[7], print_list[7] = _summarize(0, maxDets=self.params.maxDets[1])
        stats[8], print_list[8] = _summarize(0, maxDets=self.params.maxDets[2])
        stats[9], print_list[9] = _summarize(0, areaRng='small', maxDets=self.params.maxDets[2])
        stats[10], print_list[10] = _summarize(0, areaRng='medium', maxDets=self.params.maxDets[2])
        stats[11], print_list[11] = _summarize(0, areaRng='large', maxDets=self.params.maxDets[2])

        if not self.eval:
            raise Exception('Please run accumulate() first')

        return stats

修改模型文件中的coco_eval.py文件，例如：Deformable DETR中的 Deformable-DETR-main\datasets\coco_eval.py。

在summarize方法中调用pycocotools中的summarize_2方法，获取每个类别的AP0.5值，并将其保存在stats.txt文本中。

其中classIds用于定义当前数据集中类别的名字及个数。

    def summarize(self):
        for iou_type, coco_eval in self.coco_eval.items():
            print("IoU metric: {}".format(iou_type))
            coco_eval.summarize()
                
        class_AP_list = []
        classIds = ["a", "b", "c", "d","e","f","g","h","i","j"]
        for i in range(10):
            stats = coco_eval.summarize_2(i)
            class_AP_list.append(" {:15}: {}".format(classIds[i], stats[1]))

        class_AP_info = "\n".join(class_AP_list)

        # 将验证结果保存至txt文件中
        with open('stats.txt', 'a') as f:
            record_lines = ["mAP(IoU=0.5) for each category:", class_AP_info]
            f.write("\n".join(record_lines))

获取各个类别的IoU=0.5的APS、APM及APL

修改pycocotools包中的summarize方法
将stats的大小扩充为15，用于存储IoU=0.5时的APS、APM及APL

具体代码如下:

   def summarize(self):
        '''
        Compute and display summary metrics for evaluation results.
        Note this functin can *only* be applied on the default parameter setting
        '''
        def _summarize( ap=1, iouThr=None, areaRng='all', maxDets=100 ):
            p = self.params
            iStr = ' {:<18} {} @[ IoU={:<9} | area={:>6s} | maxDets={:>3d} ] = {:0.3f}'
            titleStr = 'Average Precision' if ap == 1 else 'Average Recall'
            typeStr = '(AP)' if ap==1 else '(AR)'
            iouStr = '{:0.2f}:{:0.2f}'.format(p.iouThrs[0], p.iouThrs[-1]) \
                if iouThr is None else '{:0.2f}'.format(iouThr)

            aind = [i for i, aRng in enumerate(p.areaRngLbl) if aRng == areaRng]
            mind = [i for i, mDet in enumerate(p.maxDets) if mDet == maxDets]
            if ap == 1:
                # dimension of precision: [TxRxKxAxM]
                s = self.eval['precision']
                # IoU
                if iouThr is not None:
                    t = np.where(iouThr == p.iouThrs)[0]
                    s = s[t]
                s = s[:,:,:,aind,mind]
            else:
                # dimension of recall: [TxKxAxM]
                s = self.eval['recall']
                if iouThr is not None:
                    t = np.where(iouThr == p.iouThrs)[0]
                    s = s[t]
                s = s[:,:,aind,mind]
            if len(s[s>-1])==0:
                mean_s = -1
            else:
                mean_s = np.mean(s[s>-1])
            print(iStr.format(titleStr, typeStr, iouStr, areaRng, maxDets, mean_s))
            return mean_s
        def _summarizeDets():
            stats = np.zeros((15,))
            stats[0] = _summarize(1)
            stats[1] = _summarize(1, iouThr=.5, maxDets=self.params.maxDets[2])
            stats[2] = _summarize(1, iouThr=.75, maxDets=self.params.maxDets[2])
            stats[3] = _summarize(1, areaRng='small', maxDets=self.params.maxDets[2])
            stats[4] = _summarize(1, areaRng='medium', maxDets=self.params.maxDets[2])
            stats[5] = _summarize(1, areaRng='large', maxDets=self.params.maxDets[2])
            stats[6] = _summarize(0, maxDets=self.params.maxDets[0])
            stats[7] = _summarize(0, maxDets=self.params.maxDets[1])
            stats[8] = _summarize(0, maxDets=self.params.maxDets[2])
            stats[9] = _summarize(0, areaRng='small', maxDets=self.params.maxDets[2])
            stats[10] = _summarize(0, areaRng='medium', maxDets=self.params.maxDets[2])
            stats[11] = _summarize(0, areaRng='large', maxDets=self.params.maxDets[2])
            stats[12] = _summarize(1, iouThr=.5, areaRng='small', maxDets=self.params.maxDets[2])
            stats[13] = _summarize(1, iouThr=.5, areaRng='medium', maxDets=self.params.maxDets[2])
            stats[14] = _summarize(1, iouThr=.5, areaRng='large', maxDets=self.params.maxDets[2])
            return stats
        def _summarizeKps():
            stats = np.zeros((10,))
            stats[0] = _summarize(1, maxDets=20)
            stats[1] = _summarize(1, maxDets=20, iouThr=.5)
            stats[2] = _summarize(1, maxDets=20, iouThr=.75)
            stats[3] = _summarize(1, maxDets=20, areaRng='medium')
            stats[4] = _summarize(1, maxDets=20, areaRng='large')
            stats[5] = _summarize(0, maxDets=20)
            stats[6] = _summarize(0, maxDets=20, iouThr=.5)
            stats[7] = _summarize(0, maxDets=20, iouThr=.75)
            stats[8] = _summarize(0, maxDets=20, areaRng='medium')
            stats[9] = _summarize(0, maxDets=20, areaRng='large')
            return stats
        if not self.eval:
            raise Exception('Please run accumulate() first')
        iouType = self.params.iouType
        if iouType == 'segm' or iouType == 'bbox':
            summarize = _summarizeDets
        elif iouType == 'keypoints':
            summarize = _summarizeKps
        self.stats = summarize()

效果如下：

若想将每一轮的评价指标保存在stats.txt文本中，可以将如下代码追加到模型文件中的coco_eval.py文件的summarize方法中

        stats = coco_eval.stats
        my_dict = {}
        my_dict["AP0.5:0.95all"] = stats[0]
        my_dict["AP0.5all"] = stats[1]
        my_dict["AP0.75all"] = stats[2]
        my_dict["AP0.5:0.95small"] = stats[3]
        my_dict["AP0.5:0.95medium"] = stats[4]
        my_dict["AP0.5:0.95large"] = stats[5]
        my_dict["AR0.5:0.95allmax1"] = stats[6]
        my_dict["AR0.5:0.95allmax10"] = stats[7]
        my_dict["AR0.5:0.95allmax100"] = stats[8]
        my_dict["AR0.5:0.95small"] = stats[9]
        my_dict["AR0.5:0.95medium"] = stats[10]
        my_dict["AR0.5:0.95large"] = stats[11]
        my_dict["AP0.5small"] = stats[12]
        my_dict["AP0.5medium"] = stats[13]
        my_dict["AP0.5large"] = stats[14]
        with open('stats.txt', 'a') as file:
            # 在文件末尾追加写入数据
            # 遍历字典中的键值对，将它们写入文件
            file.write('\n')
            for key, value in my_dict.items():
                file.write(f"{key}: {value}\n")
            file.write('\n')
            file.write('\n')

为了辨别stats.txt输出的评价指标属于哪一轮，可以在epoch的循环处往stats.txt文本写入当前的轮数。

stats.txt效果如下：