引言
做完K-means的改进之后,再试着从NMS入手。
NMS
首先看一下NMS算法的伪代码:
- 集合B表示候选框集合,集合S表示对应候选框得分集合,集合D表示最终结果集合;
- 在B中选出得分最高相应的候选框,记为M。在最终结果集合D中添加M并且在集合B中删除M;
- 计算B中其余候选框与M的交并比(IoU)值,将B中所有交并比值大于重叠阈值Nt的候选框删除;
- 重复步骤(2)和(3)直到集合B为空。
其中步骤(3)所用表达式如图所示:
其中,si表示候选框i的得分,当前得分最高的候选框为M,iou(M,bi)为候选框bi与M的交并比。当交并比大于等于超参数设定的重叠阈值Nt时,候选框i的得分便直接设为0,即相当于删除。
因此可知,NMS算法对于重叠阈值Nt设定敏感,设定过低会导致漏检,设定过高会导致误检。如下图所示,两匹马的得分分别为0.95和0.80,若按照NMS算法处理,首先会选中得分为0.95的框,然后得分为0.80的候选框会由于交并比过高而删除。
源代码
Soft-NMS
首先看一眼Soft-NMS的伪代码:
可以看出两者主要过程都是一样的,只不过Soft-NMS并不是将大于阈值的置信度直接变为0,而是通过函数进行改变。也就是有更大的容错率。
这里提供线性和高斯两种函数:
由于线性不连续,所以高斯用的更多。
先说结论:Soft-NMS只针对上图那两匹马一样的效果较好,即重叠物体,同时带来问题容易错检
源代码
# coding: utf-8
from __future__ import division, print_function
import numpy as np
import tensorflow as tf
def py_nms(boxes, scores, max_boxes=50, iou_thresh=0.5):
"""
Pure Python NMS baseline.
Arguments: boxes: shape of [-1, 4], the value of '-1' means that dont know the
exact number of boxes
scores: shape of [-1,]
max_boxes: representing the maximum of boxes to be selected by non_max_suppression
iou_thresh: representing iou_threshold for deciding to keep boxes
"""
assert boxes.shape[1] == 4 and len(scores.shape) == 1
x1 = boxes[:, 0]
y1 = boxes[:, 1]
x2 = boxes[:, 2]
y2 = boxes[:, 3]
areas = (x2 - x1) * (y2 - y1)
order = scores.argsort()[::-1]
keep = []
while order.size > 0:
i = order[0]
keep.append(i)
xx1 = np.maximum(x1[i], x1[order[1:]])
yy1 = np.maximum(y1[i], y1[order[1:]])
xx2 = np.minimum(x2[i], x2[order[1:]])
yy2 = np.minimum(y2[i], y2[order[1:]])
w = np.maximum(0.0, xx2 - xx1 + 1)
h = np.maximum(0.0, yy2 - yy1 + 1)
inter = w * h
ovr = inter / (areas[i] + areas[order[1:]] - inter)
inds = np.where(ovr <= iou_thresh)[0]
order = order[inds + 1]
return keep[:max_boxes]
def cpu_nms(boxes, scores, num_classes, max_boxes=50, score_thresh=0.5, iou_thresh=0.5):
"""
Perform NMS on CPU.
Arguments:
boxes: shape [1, 10647, 4]
scores: shape [1, 10647, num_classes]
"""
boxes = boxes.reshape(-1, 4)
scores = scores.reshape(-1, num_classes)
# Picked bounding boxes
picked_boxes, picked_score, picked_label = [], [], []
for i in range(num_classes):
indices = np.where(scores[:,i] >= score_thresh)
filter_boxes = boxes[indices]
filter_scores = scores[:,i][indices]
if len(filter_boxes) == 0:
continue
# do non_max_suppression on the cpu
indices = py_nms(filter_boxes, filter_scores,
max_boxes=max_boxes, iou_thresh=iou_thresh)
picked_boxes.append(filter_boxes[indices])
picked_score.append(filter_scores[indices])
picked_label.append(np.ones(len(indices), dtype='int32')*i)
if len(picked_boxes) == 0:
return None, None, None
boxes = np.concatenate(picked_boxes, axis=0)
score = np.concatenate(picked_score, axis=0)
label = np.concatenate(picked_label, axis=0)
return boxes, score, label