FSAF 模型源码解读(tensorflow+keras )

FSAF 核心思想

    FSAF全名：Feature Selective Anchor-Free Module for Single-Shot Object Detection。

    FSAF 模型创新点在于无锚框特征选择模块（FSAF)，设计了无锚框分支网络，训练过程使得模型能在金字塔特征层中选择合适的特征层，进行回归与分类，而现有模型大多基于featrue map size大小设定anchor大小，然后进行回归于分类。FSAF 模块可以添加于所有单步类FPN网络，如RetinaNet, YOLOv3等。整体网络结构为论文中Figure 4：

    FSAF 在线特征选择模块思路：首先实例分配到所有特征层，计算所有特征层中focal loss, ioU losss,选取所有层中两者后最小的层作为特征层，在选择特征层上计算分类于回归损失以进行训练，损失的计算作者设计了有效区域(efficinet region)，以及忽视区域（ignore region），计算损失时只考虑有效区域的损失。见论文Figure 6所示。 

   这样的设计方式，本人认为更加符合逻辑推理。同时作者也给出了对比，在大部分情况下，自动选择的特征与基于featrue map大小选择的特征层相同，但是也有不同的层差异。具体可以参看论文中fig.8对比结果。

源码解读

   本文就FSAF 源码解读进行记录，一方面可以增加自己对论文的理解，也可以学习优秀代码提升代码能力。该源码以tensorflow+keras为框架，在Retinanet,YOLOv3 基础上结合FSAF模块的代码，上述两个网络的代码基于keras进行搭建，具体可以参照源码。解读部分重点记录FSAF模块，包括特征选择层，损失函数计算层，推理部分的bounding box 的计算方法。

源码地址：https://github.com/xuannianz/FSAF.git

---

def fsaf(
        inputs,
        backbone_layers,
        num_classes,
        create_pyramid_features=__create_pyramid_features,
        name='fsaf'
):
    """
    Construct a RetinaNet model on top of a backbone.

    This model is the minimum model necessary for training (with the unfortunate exception of anchors as output).

    Args
        inputs: keras.layers.Input (or list of) for the input to the model.
        num_classes: Number of classes to classify.
        num_anchors: Number of base anchors.
        create_pyramid_features : Functor for creating pyramid features given the features C3, C4, C5 from the backbone.
        submodels: Submodels to run on each feature map (default is regression and classification submodels).
        name: Name of the model.

    Returns
        A keras.models.Model which takes an image as input and outputs generated anchors and the result from each submodel on every pyramid level.

        The order of the outputs is as defined in submodels:
        ```
        [
            regression, classification, other[0], other[1], ...
        ]
        ```
    """
    image_input = inputs[0]
    gt_boxes_input = inputs[1]
    feature_shapes_input = inputs[2]
    submodels = default_fsaf_submodels(num_classes)

    C3, C4, C5 = backbone_layers

    # compute pyramid features as per https://arxiv.org/abs/1708.02002
    # [P3, P4, P5, P6, P7]
    features = create_pyramid_features(C3, C4, C5)
    # for all pyramid levels, run available submodels
    # [(b, sum(fh*fw), 4), (b, sum(fh*fw), num_classes)]
    batch_regr_pred, batch_cls_pred = __build_fsaf_pyramid(submodels, features)
    batch_gt_box_levels = LevelSelect(name='level_select')(
        [batch_cls_pred, batch_regr_pred, feature_shapes_input, gt_boxes_input])
    batch_cls_target, batch_cls_mask, batch_cls_num_pos, batch_regr_target, batch_regr_mask = FSAFTarget(
        num_classes=num_classes,
        name='fsaf_target')(
        [batch_gt_box_levels, feature_shapes_input, gt_boxes_input])
    focal_loss_graph = focal_with_mask()
    iou_loss_graph = iou_with_mask()
    cls_loss = keras.layers.Lambda(focal_loss_graph,
                                   output_shape=(1,),
                                   name="cls_loss")(
        [batch_cls_target, batch_cls_pred, batch_cls_mask, batch_cls_num_pos])
    regr_loss = keras.layers.Lambda(iou_loss_graph,
                                    output_shape=(1,),
                                    name="regr_loss")([batch_regr_target, batch_regr_pred, batch_regr_mask])
    return keras.models.Model(inputs=inputs,
                              outputs=[cls_loss, regr_loss, batch_cls_pred, batch_regr_pred],
                              name=name)

该函数在backbone基础上添加了FSAF模块。

1.features = create_pyramid_features(C3, C4, C5) 创建金字塔特征层。

2.batch_regr_pred, batch_cls_pred = __build_fsaf_pyramid(submodels, features)在金字塔层添加bounding box、分类子模块。

3 batch_gt_box_levels = LevelSelect(name='level_select') 构建在线特征选择层。

4.batch_regr_mask = FSAFTarget(
        num_classes=num_classes,
        name='fsaf_target')

根据选择的特征层构建监督信号，进行训练。即根据选定的层生成包含 efficient region区域，计算损失。