【tensorflow + Faster RCNN】损失函数构建

前面两篇博客已经叙述了基于vgg模型构建faster rcnn的过程：

tensorflow+faster rcnn代码理解（一）：构建vgg前端和RPN网络

tensorflow+faster rcnn代码解析（二）：anchor_target_layer、proposal_target_layer、proposal_layer

 faster rcnn的Loss分为4部分：

1. RPN class loss

构成RPN分类损失的输入如下，采用softmaxloss：

• rpn_cls_score_reshape：训练过程中anchor预测为前景或者背景的分数；大小为（1,342,50,2）
• rpn_label：anchor被标注的类别，前景or背景 ；大小为(1,1,342,50)

# RPN, class loss
rpn_cls_score = tf.reshape(self._predictions['rpn_cls_score_reshape'], [-1, 2]) #rpn_cls_score = （17100,2）
rpn_label = tf.reshape(self._anchor_targets['rpn_labels'], [-1])  #rpn_label = (17100,)
rpn_select = tf.where(tf.not_equal(rpn_label, -1)) #将不等于-1的labels选出来（也就是正负样本选出来）,返回序号
rpn_cls_score = tf.reshape(tf.gather(rpn_cls_score, rpn_select), [-1, 2]) #同时选出对应的分数  大小为(256,2)  这里256相当于batch
rpn_label = tf.reshape(tf.gather(rpn_label, rpn_select), [-1]) #选出对应的label  大小为(256,1)  这里256相当于batch
rpn_cross_entropy = tf.reduce_mean(
                    tf.nn.sparse_softmax_cross_entropy_with_logits(logits=rpn_cls_score, labels=rpn_label))

2. RPN bbox loss

构成RPNbbox回归损失的输入如下，采用SmoothL1Loss：

• rpn_bbox_pred：训练过程中预测的anchor偏移量
• rpn_bbox_targets：anchor与gt实际的偏移量
• rpn_bbox_inside_weights：anchor内权重
• rpn_bbox_outside_weights：anchor外权重

# RPN, bbox loss
rpn_bbox_pred = self._predictions['rpn_bbox_pred']
rpn_bbox_targets = self._anchor_targets['rpn_bbox_targets']
rpn_bbox_inside_weights = self._anchor_targets['rpn_bbox_inside_weights']
rpn_bbox_outside_weights = self._anchor_targets['rpn_bbox_outside_weights']

rpn_loss_box = self._smooth_l1_loss(rpn_bbox_pred, rpn_bbox_targets, rpn_bbox_inside_weights,
rpn_bbox_outside_weights, sigma=sigma_rpn, dim=[1, 2, 3])

3. RCNN class loss

构成RCNN分类损失的输入如下，采用softmaxloss：

• cls_score：训练过程中fast rcnn部分rois被预测的类别
• labels：rois被标注的类别，属于哪一类

# RCNN, class loss
cls_score = self._predictions["cls_score"]
label = tf.reshape(self._proposal_targets["labels"], [-1])

cross_entropy = tf.reduce_mean(tf.nn.sparse_softmax_cross_entropy_with_logits(
                logits=tf.reshape(cls_score, [-1, self._num_classes]), labels=label))

4 RCNN bbox loss

构成RCNNbbox回归损失的输入如下，采用SmoothL1Loss：

• bbox_pred：训练过程中fast rcnn部分rois预测的偏移量
• bbox_targets：rois与gt实际的偏移量
• bbox_inside_weights：rois内权重
• bbox_outside_weights：rois外权重

# RCNN, bbox loss
bbox_pred = self._predictions['bbox_pred'] #(128,12)
bbox_targets = self._proposal_targets['bbox_targets'] #(128,12)
bbox_inside_weights = self._proposal_targets['bbox_inside_weights']#(128,12)
bbox_outside_weights = self._proposal_targets['bbox_outside_weights']#(128,12)

loss_box = self._smooth_l1_loss(bbox_pred, bbox_targets, bbox_inside_weights, bbox_outside_weights)