目标检测 - IoU和GIoU作为边框回归的损失和代码实现

目标检测 - IoU和GIoU作为边框回归的损失和代码实现

flyfish

GIoU
=General-IOU
=Generalized Intersection over Union

论文《Generalized Intersection over Union: A Metric and A Loss for Bounding Box Regression》

IoU和GIoU作为边框回归的损失

GIoU as Loss for Bounding Box Regression
算法过程如下

输入: 预测的边框 $B^p$ 和 GT边框 $B^g$ 的坐标
$B^p=(x_1^p,y_1^p,x_2^p,y_2^p)$, $B^g=(x_1^g,y_1^g,x_2^g,y_2^g)$
输出: ${\mathcal{L}}_{IoU}$, ${\mathcal{L}}_{GIoU}$

1. 对于预测的边框 $B^p$, 确保 $x_2^p>x_1^p$ ， $y_2^p>y_1^p$:
   ${\hat{x}}_1^p=\min (x_1^p,x_2^p)$,
   ${\hat{x}}_2^p=\max (x_1^p,x_2^p)$,
   ${\hat{y}}_1^p=\min (y_1^p,y_2^p)$,
   ${\hat{y}}_2^p=\max (y_1^p,y_2^p)$
2. 计算$B^g$的面积: $A^g=(x_2^g-x_1^g)\times (y_2^g-y_1^g)$
3. 计算$B^p$的面积: $A^p=({\hat{x}}_2^p-{\hat{x}}_1^p)\times ({\hat{y}}_2^p-{\hat{y}}_1^p)$
4. 在 $B^p$ 和 $B^g$之间计算交集$\mathcal{I}$ :
   $x_1^{\mathcal{I}}=\max ({\hat{x}}_1^p,x_1^g)$,
   $x_2^{\mathcal{I}}=\min ({\hat{x}}_2^p,x_2^g)$,
   $y_1^{\mathcal{I}}=\max ({\hat{y}}_1^p,y_1^g)$,
   $y_2^{\mathcal{I}}=\min ({\hat{y}}_2^p,y_2^g)$,
   $\mathcal{I}=\{\begin{array}{ll}(x_2^{\mathcal{I}}-x_1^{\mathcal{I}})\times (y_2^{\mathcal{I}}-y_1^{\mathcal{I}})& \text{if}{x}_2^{\mathcal{I}}>x_1^{\mathcal{I}},y_2^{\mathcal{I}}>y_1^{\mathcal{I}},\\ 0& \text{otherwise}\end{array}$
5. 寻找最小封闭框（smallest enclosing box，可看下图更清楚）的坐标 $B^c$:
   $x_1^c=\min ({\hat{x}}_1^p,x_1^g)$,
   $x_2^c=\max ({\hat{x}}_2^p,x_2^g)$,
   $y_1^c=\min ({\hat{y}}_1^p,y_1^g)$,
   $y_2^c=\max ({\hat{y}}_2^p,y_2^g)$
6. 计算$B^c$的面积: $A^c=(x_2^c-x_1^c)\times (y_2^c-y_1^c)$
7. $IoU=\frac{\mathcal{I}}{\mathcal{U}}$, where $\mathcal{U}=A^p+A^g-\mathcal{I}$
8. $GIoU=IoU-\frac{A^c-\mathcal{U}}{A^c}$
9. ${\mathcal{L}}_{IoU}=1-IoU$, ${\mathcal{L}}_{GIoU}=1-GIoU$

用图说明的更清楚
IoU=Jaccard Index

最小的封闭框是如何计算的

在 $B^p$ 和 $B^g$之间计算黄色的交集$\mathcal{I}$，绿色边框表示最小的封闭框$B^c$。
最小封闭框=C

代码实现

关于IoU Loss

根据论文UnitBox和论文GIoU对与IoU Loss处理不同的方法
UnitBox的是-ln(IoU) ln是以e为底的对数

图上的坐标tblr的表示方式是这里的第三种Center-Size coordinates
$$y_i={\log }_e(x_i)$$
输出是$x_i$ 输出是$y_i$

GIoU里的是1-IoU
所以代码实现的时候，可以同时实现三个Loss
参考
UnitBox: An Advanced Object Detection Network

代码中的坐标表示方法采用了这里中的第一种boundary coordinates (x_min, y_min, x_max, y_max)
0,1,2,3下标可表示left，top，right，bottom
x2>x1,y2>y1

import torch
import torch.nn as nn
class IoULoss(nn.Module):
    """
    Intersetion Over Union (IoU) loss 支持三种不同的loss计算方法:
    * IoU(UnitBox paper)
    * Linear IoU(GIoU paper)
    * gIoU
    * 类型支持：iou,linear_iou,giou
    """
    def __init__(self, loc_loss_type='giou'):
        super(IoULoss, self).__init__()
        self.loc_loss_type = loc_loss_type

    def forward(self, pred, gt, weight=None):
        """
        Args:
            pred: Nx4 predicted bounding boxes, Each row is (x1, y1, x2, y2).
            gt: Nx4 gt bounding boxes
        """
        pred_x1 = pred[:, 0]
        print(pred_x1)
        pred_y1 = pred[:, 1]
        pred_x2 = pred[:, 2]
        pred_y2 = pred[:, 3]

        gt_x1 = gt[:, 0]
        gt_y1 = gt[:, 1]
        gt_x2 = gt[:, 2]
        gt_y2 = gt[:, 3]
        #如果再严谨些,代码确保x2>x1,y2>y1，下标0和下标2，谁小谁是x1

        gt_aera = (gt_x1 + gt_x2) * (gt_y1 + gt_y2) #对应算法第2步
        pred_aera = (pred_x1 + pred_x2) * (pred_y1 + pred_y2)#对应算法第3步

        I_x1 = torch.max(pred_x1, gt_x1)
        I_x2 = torch.min(pred_x2, gt_x2)
        I_y1 = torch.max(pred_y1, gt_y1)
        I_y2 = torch.min(pred_x2, gt_x2)
        area_intersect=(I_x2 - I_x1)*(I_y2-I_y1)#交集 对应算法第4步

        C_x1 = torch.min(pred_x1, gt_x1)
        C_x2 = torch.max(pred_x2, gt_x2)
        C_y1 = torch.min(pred_y1, gt_y1)
        C_y2 = torch.max(pred_x2, gt_x2)
        ac =(C_x2 - C_x1) * (C_y2 - C_y1)#最小封闭框 #对应算法第5步

        U = gt_aera + pred_aera - area_intersect#并集

        ious = (area_intersect ) / (U.clamp(min=1e-10))#分母不为0
        gious = ious - (ac - U) / ac.clamp(min=1e-10)
        if self.loc_loss_type == 'iou':
            losses = -torch.log(ious)
        elif self.loc_loss_type == 'linear_iou':
            losses = 1 - ious
        elif self.loc_loss_type == 'giou':
            losses = 1 - gious
        else:
            raise NotImplementedError

        if weight is not None:
            return (losses * weight).sum()
        else:
            return losses.sum()

GIoU官方实现代码