ssd检测模型中的hard example mining源码解析以及这一机制对训练loss的影响

计算机视觉交流群：677855967，欢迎大家加入交流。

最近尝试用coco数据集训练tensorflow object detection api(下简称api)中的ssd检测模型；在tensorboard中监测到loss曲线如下：

对总体loss进行局部放大如下：                                                                   

 

由loss曲线可知，loss是一直在下降中，但其中有些步数的loss会出现一个高峰，仔细观察发现，这个高峰出现的频率居然和hardexample miner中negativenum以及positive num的峰值是重合的，那他们到底是和关系？带着这个问题先去翻阅了一下论文中对于hardnegative mining的介绍：很简短的一段文字。

看完这段文字对与negative基本能有个大概认知，然而对于实现细节还是存疑，继续翻阅api中ssd检测源码中的hardexampleminer的源码，先把代码贴出来：

def build_hard_example_miner(config,
                             classification_weight,
                             localization_weight):
  """Builds hard example miner based on the config.

  Args:
    config: A losses_pb2.HardExampleMiner object.
    classification_weight: Classification loss weight.
    localization_weight: Localization loss weight.

  Returns:
    Hard example miner.

  """
  loss_type = None
  if config.loss_type == losses_pb2.HardExampleMiner.BOTH:
    loss_type = 'both'
  if config.loss_type == losses_pb2.HardExampleMiner.CLASSIFICATION:
    loss_type = 'cls'
  if config.loss_type == losses_pb2.HardExampleMiner.LOCALIZATION:
    loss_type = 'loc'

  max_negatives_per_positive = None
  num_hard_examples = None
  if config.max_negatives_per_positive > 0:
    max_negatives_per_positive = config.max_negatives_per_positive
  if config.num_hard_examples > 0:
    num_hard_examples = config.num_hard_examples
  hard_example_miner = losses.HardExampleMiner(
      num_hard_examples=num_hard_examples,
      iou_threshold=config.iou_threshold,
      loss_type=loss_type,
      cls_loss_weight=classification_weight,
      loc_loss_weight=localization_weight,
      max_negatives_per_positive=max_negatives_per_positive,
      min_negatives_per_image=config.min_negatives_per_image)
  return hard_example_miner

这个是api中对hardexample miner的源码：其中我们看到mining方式分为三种方式：分类，检测，和这两种组合共三种；

config：指定三种方式中使用哪一种，negative和positive比例，最大的hardexample数量等，如下所示；

使用这些参数初始化了HardExampleMiner（）这样一个object；源码如下：

class HardExampleMiner(object):
  """Hard example mining for regions in a list of images.

  Implements hard example mining to select a subset of regions to be
  back-propagated. For each image, selects the regions with highest losses,
  subject to the condition that a newly selected region cannot have
  an IOU > iou_threshold with any of the previously selected regions.
  This can be achieved by re-using a greedy non-maximum suppression algorithm.
  A constraint on the number of negatives mined per positive region can also be
  enforced.

  Reference papers: "Training Region-based Object Detectors with Online
  Hard Example Mining" (CVPR 2016) by Srivastava et al., and
  "SSD: Single Shot MultiBox Detector" (ECCV 2016) by Liu et al.
  """

  def __init__(self,
               num_hard_examples=64,
               iou_threshold=0.7,
               loss_type='both',
               cls_loss_weight=0.05,
               loc_loss_weight=0.06,