图像处理随笔——非极大值抑制

本篇论文主要是针对目标检测中的非极大值抑制这部分做个简单叙述。在目标检测的任务中，非极大值抑制（NMS）是一种对检测结果进行冗余去除操作的后处理算法，可以理解为局部最大搜索。标准的极大值抑制算法是手工设计一个阈值，基于这个固定的距离值进行贪婪聚类 ，即：贪婪的选取得分高的检测结果并删除那些超过阈值的相邻结果，使得在recall和precision之间取得权衡。NMS是一个迭代-遍历-消除的过程。

（1）将所有框的得分排序，选中最高分及其对应的框

（2）遍历其余的框，如果和当前最高分框的重叠面积大于一定阈值，我们就将框删除。

（3）从未处理的框中继续选一个得分最高的，重复上述过程

代码如下：

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
"""
Created on Wednesday August 14 15:50:00 2018

@author: CW
"""
import numpy as np
import matplotlib.pyplot as plt


boxes = np.array([[100, 100, 210, 210, 0.71],
                  [250, 250, 420, 420, 0.8],
                  [220, 200, 320, 330, 0.92],
                  [100, 100, 210, 210, 0.72],
                  [230, 240, 325, 330, 0.81],
                  [220, 230, 315, 340, 0.9]])
'''
boxes = np.array([[3, 6, 9, 11, 0.9],
                  [6, 3, 8, 7, 0.6],
                  [3, 7, 10, 12, 0.7],
                  [1, 4, 13, 7, 0.2]])
'''

def iou(xmin, ymin, xmax, ymax, areas, lastInd, beforeInd, threshold):
    # 将lastInd指向的box，与之前的所有存活的box指向坐标做比较
    xminTmp = np.maximum(xmin[lastInd], xmin[beforeInd])
    yminTmp = np.maximum(ymin[lastInd], ymin[beforeInd])
    xmaxTmp = np.minimum(xmax[lastInd], xmax[beforeInd])
    ymaxTmp = np.minimum(ymax[lastInd], ymax[beforeInd])

    # 计算lastInd指向的box，与其他box交集的，所有width，height
    width = np.maximum(0.0, xmaxTmp - xminTmp + 1)
    height = np.maximum(0.0, ymaxTmp - yminTmp + 1)

    # 计算存活box与last指向box的交集面积
    intersection = width * height
    union = areas[beforeInd] + areas[lastInd] - intersection
    iou_value = intersection / union

    indexOutput = [item[0] for item in zip(beforeInd, iou_value) if item[1] <= threshold]

    return indexOutput

def nms(boxes, threshold):
    assert isinstance(boxes, np.ndarray)
    assert boxes.shape[1] == 5

    xmin = boxes[:, 0]
    ymin = boxes[:, 1]
    xmax = boxes[:, 2]
    ymax = boxes[:, 3]
    scores = boxes[:, 4]

    # calc area of each box
    areas = (xmax - xmin + 1) * (ymax - ymin + 1)

    # score each box in ascending order
    scoresSorted = sorted(list(enumerate(scores)), key=lambda item: item[1])
    # save index
    index = [item[0] for item in scoresSorted]

    keep = []
    while len(index) > 0:
        lastInd = index[-1]
        keep.append(lastInd)

        # calc the iou of the last box and all the boxes before it
        index = iou(xmin, ymin, xmax, ymax, areas, lastInd, index[:-1], threshold)
    return keep

def bbox(dets, c='k'):
    x1 = dets[:, 0]
    y1 = dets[:, 1]
    x2 = dets[:, 2]
    y2 = dets[:, 3]

    plt.plot([x1, x2], [y1, y1], c)
    plt.plot([x1, x1], [y1, y2], c)
    plt.plot([x1, x2], [y2, y2], c)
    plt.plot([x2, x2], [y1, y2], c)
    plt.title("after nms")

if __name__ == '__main__':
    # before nms
    bbox(boxes, 'k')
    remain = nms(boxes, threshold=0.6)

    # after nms
    bbox(boxes[remain], 'r')

非极大值抑制之前：

非极大值抑制之后：

贪心NMS存在的问题：

1. 当两个目标框接近时，分数更低的框就会因为与之重叠面积过大而被删掉。

2. NMS的阈值也不太容易确定，设小了漏检，设置过高容易增大误检。