darknet yolo 计算mAP，recall

2017 year:

1.生成检测结果文件

./darknet detector valid cfg/car.data cfg/car.cfg backup/car_final.weights -out car.txt -gpu 0 -thresh .5

2.把car.txt 用faster rcnn 中voc_eval计算mAP

/home/sam/src/caffeup2date_pyfasterrcnn/lib/datasets/compute_mAP.py

from voc_eval import voc_eval

print voc_eval('/home/sam/src/darknet/results/{}.txt', '/home/sam/datasets/car2/VOC2007/Annotations/{}.xml', '/home/sam/datasets/car2/VOC2007/ImageSets/Main/test.txt', 'car', '.')

第三个结果就是

 

如果只想计算大于0.3的输出结果的mAP,把 voc_eval.py文件中如下代码更改

sorted_ind = np.argsort(-confidence)
sorted_ind1 = np.where(confidence[sorted_ind] >= .3)[0]#np.argsort(-confidence<=-.3)
sorted_ind = sorted_ind[sorted_ind1]

 

 

3.计算recall 

./darknet detector recall cfg/car.data cfg/car.cfg backup/car_final.weights -out car.txt -gpu 0 -thresh .5

---

2020.12 更新

1. darknet方式

这个仓库，对map计算做了新的函数，可以非常方便的使用。另外上面说的计算recall 是不对的，只是计算了object的recall.

https://github.com/AlexeyAB/darknet

./darknet detector map  ./cfg/voc-kitchen.data ./cfg/yolov3-kitchen.cfg ./backup/yolov3-kitchen.backup -gpu 0 -thresh 0.25 -map

Total BFLOPS 197.094 
avg_outputs = 1559942 
 Allocate additional workspace_size = 52.43 MB 
Loading weights from ./backup/yolov3-kitchen.backup...
 seen 64, trained: 377 K-images (5 Kilo-batches_64) 
Done! Loaded 107 layers from weights-file 

 calculation mAP (mean average precision)...
 Detection layer: 82 - type = 28 
 Detection layer: 94 - type = 28 
 Detection layer: 106 - type = 28 
2356
 detections_count = 16713, unique_truth_count = 12772  
class_id = 0, name = body, ap = 96.53%   	 (TP = 6061, FP = 339) 
class_id = 1, name = head, ap = 95.09%   	 (TP = 5803, FP = 335) 

 for conf_thresh = 0.25, precision = 0.95, recall = 0.93, F1-score = 0.94 
 for conf_thresh = 0.25, TP = 11864, FP = 674, FN = 908, average IoU = 75.29 % 

 IoU threshold = 50 %, used Area-Under-Curve for each unique Recall 
 mean average precision (mAP@0.50) = 0.958110, or 95.81 % 
Total Detection Time: 57 Seconds

2. 有一个仓库对map计算过程，讲解非常透彻 https://github.com/rafaelpadilla/Object-Detection-Metrics

    1. 选择一个类，做ap计算

    2.对所有图片，中该类box的检测，按照预测得分从大到小排序，

    3.按照这个顺序，计算累计tp， fp， 进而计算累计precision, recall

    4.然后按照0， 0.1， 0.2 ...  0.9 1.0 ，这11各点，10个区间，得到折线图，然后得到11个顶点，进而计算这11个顶点的平均值，就得到了，这个类的ap

    5.对所有类，计算ap， 然后对这些ap加和，除以类别数，得到mAP