2017 year:
1.生成检测结果文件
./darknet detector valid cfg/car.data cfg/car.cfg backup/car_final.weights -out car.txt -gpu 0 -thresh .5
2.把car.txt 用faster rcnn 中voc_eval计算mAP
/home/sam/src/caffeup2date_pyfasterrcnn/lib/datasets/compute_mAP.py
from voc_eval import voc_eval
print voc_eval('/home/sam/src/darknet/results/{}.txt', '/home/sam/datasets/car2/VOC2007/Annotations/{}.xml', '/home/sam/datasets/car2/VOC2007/ImageSets/Main/test.txt', 'car', '.')
第三个结果就是
如果只想计算大于0.3的输出结果的mAP,把 voc_eval.py文件中如下代码更改
sorted_ind = np.argsort(-confidence)
sorted_ind1 = np.where(confidence[sorted_ind] >= .3)[0]#np.argsort(-confidence<=-.3)
sorted_ind = sorted_ind[sorted_ind1]
3.计算recall
./darknet detector recall cfg/car.data cfg/car.cfg backup/car_final.weights -out car.txt -gpu 0 -thresh .5
2020.12 更新
1. darknet方式
这个仓库,对map计算做了新的函数,可以非常方便的使用。另外上面说的计算recall 是不对的,只是计算了object的recall.
https://github.com/AlexeyAB/darknet
./darknet detector map ./cfg/voc-kitchen.data ./cfg/yolov3-kitchen.cfg ./backup/yolov3-kitchen.backup -gpu 0 -thresh 0.25 -map
Total BFLOPS 197.094
avg_outputs = 1559942
Allocate additional workspace_size = 52.43 MB
Loading weights from ./backup/yolov3-kitchen.backup...
seen 64, trained: 377 K-images (5 Kilo-batches_64)
Done! Loaded 107 layers from weights-file
calculation mAP (mean average precision)...
Detection layer: 82 - type = 28
Detection layer: 94 - type = 28
Detection layer: 106 - type = 28
2356
detections_count = 16713, unique_truth_count = 12772
class_id = 0, name = body, ap = 96.53% (TP = 6061, FP = 339)
class_id = 1, name = head, ap = 95.09% (TP = 5803, FP = 335)
for conf_thresh = 0.25, precision = 0.95, recall = 0.93, F1-score = 0.94
for conf_thresh = 0.25, TP = 11864, FP = 674, FN = 908, average IoU = 75.29 %
IoU threshold = 50 %, used Area-Under-Curve for each unique Recall
mean average precision (mAP@0.50) = 0.958110, or 95.81 %
Total Detection Time: 57 Seconds
2. 有一个仓库对map计算过程,讲解非常透彻 https://github.com/rafaelpadilla/Object-Detection-Metrics
1. 选择一个类,做ap计算
2.对所有图片,中该类box的检测,按照预测得分从大到小排序,
3.按照这个顺序,计算累计tp, fp, 进而计算累计precision, recall
4.然后按照0, 0.1, 0.2 ... 0.9 1.0 ,这11各点,10个区间,得到折线图,然后得到11个顶点,进而计算这11个顶点的平均值,就得到了,这个类的ap
5.对所有类,计算ap, 然后对这些ap加和,除以类别数,得到mAP