评估模型
首先需要明确,AP和mAP是像素级别的
AP计算
1.更适用于判断目标是否存在的任务
个人理解是对于大量先验框,根据softmax输出的其是否包含目标的概率(置信度),即根据不同的阈值(目标存在的概率)计算精确率和置信度,来绘制精确率-置信度曲线,再通过计算曲线下面积来计算平均值
借助sklearn计算PR,结果如下:
from sklearn.metrics import precision_recall_curve
y_true = [0, 0, 1, 1]#ground truth label 真实标签,2正2负
y_score = [0.1, 0.4, 0.35, 0.8]#confidence/probability 是目标/包含目标的概率
precision, recall, thresholds = precision_recall_curve(y_true, y_score)
print(precision)
print(recall)
print(thresholds)#阈值,y_score与之作比较,区分正负样本
"""
[0.66666667 0.5 1. 1.]
[1. 0.5 0.5 0. ]
[0.35 0.4 0.8 ]
即阈值为0.35时,P为0.67,R为1;阈值为0.4时,P为0.5,R为0.5;阈值为0.8时,P为1,R为0.5
(precision和recall最后一个值分别为1和0,并且没有对应的阈值)
"""
2.适用于多目标检测/分类任务
利用sklearn 计算
from sklearn.metrics import confusion_matrix
from sklearn.metrics import classification_report
from sklearn.metrics import precision_recall_fscore_support
from sklearn.metrics import accuracy_score
actual = [0,0,0,0,1,1,0,3,3]
predicted = [0,0,0,0,1,1,2,3,3]
# 计算总的精度
acc = accuracy_score(actual, predicted)
# 计算precision, recall, F1-score, support
class_names = ['agree', 'disagree', 'discuss', 'unrelated']
print(classification_report(actual, predicted, target_names=class_names))#底层基于混淆矩阵confusion_matrix
#print(confusion_matrix(actual, predicted))
# 另一种方式计算precision, recall, F1-score, support
pre, rec, f1, sup = precision_recall_fscore_support(actual, predicted)
print("precision:", pre, "\nrecall:", rec, "\nf1-score:", f1, "\nsupport:", sup)
#两种方式计算结果相同
计算不规则目标的IOU
import numpy as np
import shapely
from shapely.geometry import Polygon,MultiPoint #多边形
line1=[2,0,2,2,0,0,0,2] #四边形四个点坐标的一维数组表示,[x,y,x,y....],无须在意方向
a=np.array(line1).reshape(4, 2) #四边形二维坐标表示
poly1 = Polygon(a).convex_hull #python四边形对象,能自动计算四个点;输出为5对,表示顺序为:左上 左下 右下 右上 左上
print(Polygon(a).convex_hull)
line2=[1,1,4,1,4,4,1,4]
b=np.array(line2).reshape(4, 2)
poly2 = Polygon(b).convex_hull
print(Polygon(b).convex_hull)
union_poly = np.concatenate((a,b)) #合并两个box坐标,变为8*2
#print(union_poly)
print(MultiPoint(union_poly).convex_hull) #包含两四边形最小的多边形点
if not poly1.intersects(poly2): #如果两四边形不相交
iou = 0
else:
try:
inter_area = poly1.intersection(poly2).area #相交面积
print(inter_area)
#union_area = poly1.area + poly2.area - inter_area
union_area = MultiPoint(union_poly).convex_hull.area
print(union_area)
if union_area == 0:
iou= 0
iou=float(inter_area) / union_area
# iou=float(inter_area) /(poly1.area+poly2.area-inter_area)
# 源码中给出了两种IOU计算方式,第一种计算的是: 交集部分/包含两个四边形最小多边形的面积
# 第二种: 交集 / 并集(常见矩形框IOU计算方式)
except shapely.geos.TopologicalError:
print('shapely.geos.TopologicalError occured, iou set to 0')
iou = 0
print(iou)
其中shapely的安装步骤为:
先在命令行输出python,查看Python版本等:
进入下载网址:
https://www.lfd.uci.edu/~gohlke/pythonlibs/#shapely
下载相应版本的shapely.whl
之后将whl文件复制到python安装目录下的scripts文件夹中
使用anaconda管理python环境的目录为:C:\Users\[用户名]\anaconda3\envs\[虚拟环境名]\Scripts
之后命令行进入该目录执行pip install [文件名.whl]
即可
计算P、R
根据上面计算的IOU,计算TP、TN、FP、FN,再计算P和R
win10安装labelImg标记数据
activate [环境名]
pip install PyQt5
pip install lxml
下载labelImg:https://github.com/tzutalin/labelImg.git
解压后进入该目录,将resources.py文件复制到libs文件夹下
cd [labelImg目录]
pyrcc5 -o resources.py resources.qrc
python labelImg.py
即可进入labelImg标记数据
Tips:
(1)可以修改labelImg/data/predefined_classes.txt文件中类别为所需类别。
(2)可以在View视图下设置以下选项方便使用:
Auto Save mode:自动保存上一张图片的标注;
Display Labels:显示标记框和标签;
Advanced Mode:默认Create RectBox,可直接多次标记。
(3)注意Change Save Dir
执行下面代码,将xml中的img路径和多个目标的[xmin,ymin],[xman,ymax]写入到一个txt文档
#将xml中的img路径和多个目标的[xmin,ymin],[xman,ymax]写入到一个txt文档中
import xml.etree.ElementTree as ET
xml_path='img.xml'
tree = ET.parse(xml_path)
rect={}
line=""
root = tree.getroot()
for name in root.iter('path'):
rect['path'] = name.text
line = rect['path']
for ob in root.iter('object'):
for bndbox in ob.iter('bndbox'):
# for l in bndbox:
# print(l.text)
for xmin in bndbox.iter('xmin'):
rect['xmin'] = xmin.text
for ymin in bndbox.iter('ymin'):
rect['ymin'] = ymin.text
for xmax in bndbox.iter('xmax'):
rect['xmax'] = xmax.text
for ymax in bndbox.iter('ymax'):
rect['ymax'] = ymax.text
line=line+ ","+ rect['xmin']+ ","+rect['ymin']+","+rect['xmax']+","+rect['ymax']
f1 = open('img.txt', 'w')
f1.write(line)