计算YOLOv3在COCO数据集上的mAP值

安装darknet

  1. 详细步骤和解释请参考YOLOv3的官网,这里给出相应的命令行和需要特别注意的地方

  2. 从github上下载项目源码
    git clone https://github.com/pjreddie/darknet
    cd darknet
  3. 对源码进行编译(编译默认运行YOLOv3时是不使用GPU的,我们假设跑YOLOv3时需要使用GPU,所以需要修改编译选项)
    1. 只需要将当前文件夹下(darknet/)的Makefile文件中第一行的GPU=0改为GPU=1,改完后的Makefile:
      GPU=1
      CUDNN=0
      OPENCV=0
      OPENMP=0
      DEBUG=0
      
      ...
      ...
      ...
    2. 编译

      make
  4. 将权重文件(yolov3.weights)下载到当前目录下
    wget https://pjreddie.com/media/files/yolov3.weights
  5. 测试
    #命令行中的‘-i 0’代表使用第0号显卡,假如不想使用GPU可以用‘-nogpu’替代‘-i 0’
    ./darknet -i 0 detector test cfg/coco.data cfg/yolov3.cfg yolov3.weights data/dog.jpg 

COCO数据集的下载和配置

  1. 通过脚本文件(scripts/get_coco_dataset.sh)获取COCO数据集(数据集下载速度会很慢,建议从Function的数据集下载站下载,然后逐行运行scripts/get_coco_dataset.sh中的每一行命令)
    cp scripts/get_coco_dataset.sh data
    cd data
    bash get_coco_dataset.sh
  2. 修改COCO的配置文件(cfg/coco.data)
    classes= 80
    train  = <path-to-coco>/trainvalno5k.txt # 注意这里的<path-to-coco>需要填入COCO数据集的文件夹路径
    valid  = <path-to-coco>/5k.txt # 注意这里的<path-to-coco>需要填入COCO数据集的文件夹路径
    names = data/coco.names
    backup = backup
    eval = coco #注意这行是为了生成json格式的检测结果,官网里并没有说,如果去掉这一行生成的文件是classes(80)类个txt文件
  3. 修改YOLOv3模型的配置文件(cfg/yolov3.cfg)

    [net]
    # Testing
    # batch=1
    # subdivisions=1
    # Training
    batch=64 #注意这里的batch有可能使得显存爆掉,这时需要修改,GeForce RTX 2080Ti需要修改为8
    subdivisions=8
    ....

对COCO验证集进行检测

  1. 在darknet目录下运行下面的命令行

    ./darknet -i 0 detector valid cfg/coco.data cfg/yolov3.cfg yolov3.weights # '-i 0'代表使用0号GPU

    运行完上述命令后会生成results/coco_results.json文件,该文件保存了检测结果

计算mAP

方法源于:https://blog.csdn.net/xidaoliang/article/details/88397280

  1. 安装pycocotools
    pip install pycocotools
  2. 在darknet目录下编写一个python脚本(compute_coco_mAP.py)用于计算mAP(注意根据自己的实际情况修改两个文件路径)

    #-*- coding:utf-8 -*-
    import matplotlib.pyplot as plt 
    from pycocotools.coco import COCO 
    from pycocotools.cocoeval import COCOeval 
    import numpy as np 
    import skimage.io as io 
    import pylab,json 
    pylab.rcParams['figure.figsize'] = (10.0, 8.0) 
    def get_img_id(file_name): 
        ls = [] 
        myset = [] 
        annos = json.load(open(file_name, 'r')) 
        for anno in annos: 
          ls.append(anno['image_id']) 
        myset = {}.fromkeys(ls).keys() 
        return myset 
    if __name__ == '__main__': 
        annType = ['segm', 'bbox', 'keypoints']#set iouType to 'segm', 'bbox' or 'keypoints'
        annType = annType[1] # specify type here
        cocoGt_file = 'data/coco/annotations/instances_val2014.json' #需要根据自己的实际情况配置该路径
        cocoGt = COCO(cocoGt_file)#取得标注集中coco json对象
        cocoDt_file = 'results/coco_results.json' #需要根据自己的实际情况配置该路径
        imgIds = get_img_id(cocoDt_file) 
        print(len(imgIds))
        cocoDt = cocoGt.loadRes(cocoDt_file)#取得结果集中image json对象
        imgIds = sorted(imgIds)#按顺序排列coco标注集image_id
        imgIds = imgIds[0:5000]#标注集中的image数据
        cocoEval = COCOeval(cocoGt, cocoDt, annType) 
        cocoEval.params.imgIds = imgIds#参数设置
        cocoEval.evaluate()#评价
        cocoEval.accumulate()#积累
        cocoEval.summarize()#总结
  3. 运行compute_coco_mAP.py脚本

    python compute_coco_mAP.py
  4. 有可能在运行compute_coco_mAP.py脚本的过程中会报错,可能的解决方法:

    1. 升级scikit-image

      pip install -U scikit-image
    2. 修改Numpy版本为1.16
      pip install numpy==1.16

结果展示

  1. img_size为608*608(img_size的设置可以通过修改cfg/yolov3.cfg文件中的width和height来实现)时:
     Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.334
     Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.585
     Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.345
     Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.194
     Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.365
     Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.439
     Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.291
     Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.446
     Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.470
     Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.304
     Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.502
     Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.593
    
  2. img_size为416*416时:
     Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.314
     Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.559
     Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.318
     Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.142
     Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.341
     Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.464
     Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.278
     Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.419
     Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.442
     Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.239
     Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.482
     Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.611
    
  3. img_size为320*320时:
     Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.286
     Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.521
     Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.284
     Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.103
     Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.315
     Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.449
     Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.260
     Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.389
     Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.408
     Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.189
     Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.454
     Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.601
    

     

关于结果的解释请参考:https://blog.csdn.net/u014734886/article/details/78831884

  • 4
    点赞
  • 27
    收藏
    觉得还不错? 一键收藏
  • 8
    评论
评论 8
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值