目标检测 Faster_RCNN Keras 版本的理解与跑通

首先，很感谢特别多博主的帖子给的启发，包括以下几篇文章，都给了我很多帮助。

https://blog.csdn.net/wangwei19871103/article/list/1?

https://zhuanlan.zhihu.com/p/28585873

https://zhuanlan.zhihu.com/p/29400164

https://blog.csdn.net/qq_34564612/article/details/78881689

 

要先跑通代码，还是要先理解一下FasterRCNN的原理吧，当然，其中最重要的是作者提出了RPN，RPN部分，这位博主的一张图描述的特别详细且适合Keras版本的代码

基本上Keras版本里面的代码照着这幅图边看边理解会有很大的帮助。

 

1. 首先是config.py，主要是一些对整个工程的一些参数设置。

from keras import backend as K


class Config:

    def __init__(self):

        self.verbose = True

        # base CNN model  特征提取阶段所用的backbone
        self.network = 'resnet50'

        # setting for data augmentation
        self.use_horizontal_flips = False
        self.use_vertical_flips = False
        self.rot_90 = False

        # anchor box scales 
        self.anchor_box_scales = [128, 256, 512]

        # anchor box ratios
        self.anchor_box_ratios = [[1, 1], [1, 2], [2, 1]]

        # size to resize the smallest side of the image
        self.im_size = 600

        # image channel-wise mean to subtract
        self.img_channel_mean = [103.939, 116.779, 123.68]
        self.img_scaling_factor = 1.0

        # number of ROIs at once
        self.num_rois = 300

        # stride at the RPN (this depends on the network configuration)
        self.rpn_stride = 16

        self.balanced_classes = False

        # scaling the stdev
        self.std_scaling = 4.0
        self.classifier_regr_std = [8.0, 8.0, 4.0, 4.0]

        # overlaps for RPN
        self.rpn_min_overlap = 0.3
        self.rpn_max_overlap = 0.7

        # overlaps for classifier ROIs
        self.classifier_min_overlap = 0.1
        self.classifier_max_overlap = 0.5

        # placeholder for the class mapping, automatically generated by the parser
        self.class_mapping = None

        #location of pretrained weights for the base network
        # weight files can be found at:
        # https://github.com/fchollet/deep-learning-models/releases/download/v0.2/resnet50_weights_th_dim_ordering_th_kernels_notop.h5
        # https://github.com/fchollet/deep-learning-models/releases/download/v0.2/resnet50_weights_tf_dim_ordering_tf_kernels_notop.h5

        self.model_path = 'model_frcnn.vgg.hdf5'  #模型保存

2. 后面我们主要看一下RPN这一部分的内容

图像输出经过特征提取的backbone之后，特征图的尺寸（长和宽）都缩小到原来的1/16. 这个参数在配置文件中也可以找到，级rnp_stride=16. 然后特征图的每个点对应原图中一片区域，也就是我们常说的感受野的大小。然后作者提出anchor的概念，也就是特征图上每个点在原图上生成9个框，这九个框的属性设置按照anchor_box_scale与anchor_box_ratio得来。示意图如下，共三种尺寸128，256，512以及三种比例[1:1]，[1:2]，[2:1]的锚框，一共得到9个框。然后我们还是从代码开始看吧

data_generators.py

由于这一部分的代码两较大，我们一步步来看，也方便我自己梳理。

from __future__ import absolute_import
import numpy as np
import cv2
import random
import copy
from . import data_augment
import threading
import itertools


def union(au, bu, area_intersection):
    area_a = (au[2] - au[0]) * (au[3] - au[1])
    area_b = (bu[2] - bu[0]) * (bu[3] - bu[1])
    area_union = area_a + area_b - area_intersection
    return area_union


def intersection(ai, bi):
    x = max(ai[0], bi[0])
    y = max(ai[1], bi[1])
    w = min(ai[2], bi[2]) - x
    h = min(ai[3], bi[3]) - y
    if w < 0 or h < 0:
        return 0
    return w*h


# Intersection of Union
def iou(a, b):
    # a and b should be (x1,y1,x2,y2)

    if a[0] >= a[2] or a[1] >= a[3] or b[0] >= b[2] or b[1] >= b[3]:
        return 0.0

    area_i = intersection(a, b)
    area_u = union(a, b, area_i)

    return float(area_i) / float(area_u + 1e-6)

这一部分主要是IOU的计算，也就是计算两个框之间的交集和并集，然后计算IOU，这里应该也比较好理解。

# image resize
def get_new_img_size(width, height, img_min_side=600):
    if width <= height:
        f = float(img_min_side) / width
        resized_height = int(f * height)
        resized_width = int(img_min_side)
    else:
        f = float(img_min_side) / height
        resized_width = int(f * width)
        resized_height = int(img_min_side)

    return resized_width, resized_height

这一部分对图像做resize，因为原始图像的gt坐标的标记是以原始图像为准的，这里这个函数还只是先将原图进行了resize，这里注意到图像的尺寸最小边的长度是600（这个数值怎么得来，是否可以选择其他值？）

接下来就是比较详细的rpn介绍部分了。

def calc_rpn(C, img_data, width, height, resized_width, resized_height, img_length_calc_function):

    downscale = float(C.rpn_stride)
    anchor_sizes = C.anchor_box_scales
    anchor_ratios = C.anchor_box_ratios
    num_anchors = len(anchor_sizes) * len(anchor_ratios)

    # calculate the output map size based on the network architecture
    (output_width, output_height) = img_length_calc_function(resized_width, resized_height)

    n_anchratios = len(anchor_ratios)

    # initialise empty output objectives
    y_rpn_overlap = np.zeros((output_height, output_width, num_anchors))
    y_is_box_valid = np.zeros((output_height, output_width, num_anchors))
    y_rpn_regr = np.zeros((output_height, output_width, num_anchors * 4))

    num_bboxes = len(img_data['bboxes'])

    num_anchors_for_bbox = np.zeros(num_bboxes).astype(int)
    best_anchor_for_bbox = -1 * np.ones((num_bboxes, 4)).astype(int)
    best_iou_for_bbox = np.zeros(num_bboxes).astype(np.float32)
    best_x_for_bbox = np.zeros((num_bboxes, 4)).astype(int)
    best_dx_for_bbox = np.zeros((num_bboxes, 4)).astype(np.float32)

    # get the GT box coordinates, and resize to account for image resizing
    gta = np.zeros((num_bboxes, 4))
    for bbox_num, bbox in enumerate(img_data['bboxes']):
        # get the GT box coordinates, and resize to account for image resizing
        gta[bbox_num, 0] = bbox['x1'] * (resized_width / float(width))
        gta[bbox_num, 1] = bbox['x2'] * (resized_width / float(width))
        gta[bbox_num, 2] = bbox['y1'] * (resized_height / float(height))
        gta[bbox_num, 3] = bbox['y2'] * (resized_height / float(height))

还是一部分一部分的来，C是配置信息，img_data包含一张图片的路径，bbox坐标和对应的分类（可能一张图片有多组，即表示图片里包含多个对象）。后面是图片的原尺寸和resize之后的尺寸，用于求bbox坐标在resize之后图片上的坐标，img_length_calc_function是一个方法，基于我们的设置来从图片尺寸计算出经过网络之后特征图的尺寸。

接下来读取了几个参数，downscale就是从图片到特征图的缩放倍数，anchor_size和anchor_ratios是我们初步选区大小的参数，比如3个size和3个ratios，可以组合成9种不同形状大小的选区。接下来通过img_.....function这个方法计算出了特征图的尺寸。

 

（未完待续，持续更新）