Bounding-box regression in RCNN/Faster-RCNN/SSD

Bounding-box regression

Basics(RCNN)

Mainly refer to appendix C in RCNN paper.

bbox regression 是一种针对bbox的机器学习回归问题

input

{ P i , G i } i = 1 , 2 , … , N \{P^i, G^i\}_{i=1,2,\dots,N} {Pi,Gi}i=1,2,…,N​

where $P^i=(P_x^i,P_y^i,P_w^i,P_h^i,)$ specifies the pixel coordinates of the center of proposal $P^i$’s bounding box together with $P^i$’s width and height in pixels.

Hence forth, we drop the superscript i unless it is needed.
Each ground-truth bounding box $G$ is specified in the same way: $G=(G_x,G_y,G_w,G_h)$

$P$ is proposal bounding box, 由于RCNN的是在region poposal的基础上做的回归，所以自然引入了$P$作为回归的初始值。同时也可以是认为是传统的滑动窗口检测思想的延伸。

goal

Our goal is to learn a transformation that maps a proposed box P to a ground-truth box $G$.

未知模型$f:P\to G$, 求$g$, $g:P\to \hat{G}$, 使$g\approx f$

model

We parameterize the transformation in terms of four functions $d_x(P),d{y}_{(}P),d_w(P),d_h(P)$. The first two specify a scale-invariant translation of the center of $P$’s bounding box, while the second two specify log-space translations of the width and height of P’s bounding box.

After learning these functions, we can transform an input proposal $P$ into a predicted ground-truth box $\hat{G}$ by applying the transformation

$$\begin{array}{rl}\widehat{G_x}& =P_w{d}_x(P)+P_w\\ \widehat{G_y}& =P_h{d}_y(P)+P_h\\ \widehat{G_w}& =P_{w}exp(d_w(P))\\ \widehat{G_h}& =P_{h}exp(d_h(P))\end{array}$$

where $d_{\ast }(P)=d_{\ast }(P,\Phi (P))=w_{\ast }^T\Phi (P)$, $\Phi (P)$ is the feature decided by $P$, $w_{\ast }^T$ is weight to be learned, ∗ ∈ { x , y , w , h } * \in \{x,y,w,h\} ∗∈{x,y,w,h}, $exp(x)=e^x$.

注意，不同于分类问题使用feature map上的所有特征，bbox regression只使用由$P$决定的局部特征。

It is easy to get
$$\begin{array}{rl}{d}_x& =({\hat{G}}_x-P_x)/P_w\\ d_y& =({\hat{G}}_y-P_y)/P_h\\ d_w& =log({\hat{G}}_w/P_w)\\ d_h& =log({\hat{G}}_h/P_h)\end{array}$$
scale-invariant translation

特征提取应该具有尺度不变性，即不同尺度的同一物体应得到相同的特征$d(P)$， 而$P$的尺度随着物体尺度变化而变化（对于RCNN)，从而尺度不变的$d_x(P),d_y(P)$能得到准确的$\hat{G}$。

log-space (width/height) translation

猜测log-space使${\delta }_w,{\delta }_h$ 与${\delta }_x,{\delta }_y$在数值上比较接近从而在loss中的贡献也比较接近。

optimize objective

$$\begin{array}{rl}{w}_{\ast }& =argmi{n}_{{\hat{w}}_{\ast }}\sum _{i=1}^{N}L({\delta }_{\ast }^i)+\lambda R({\hat{w}}_{\ast })\\ & =argmi{n}_{{\hat{w}}_{\ast }}\sum _{i=1}^{N}L[t_{\ast }^i-d_{\ast }^i(P)]+\lambda R({\hat{w}}_{\ast })\\ & =argmi{n}_{{\hat{w}}_{\ast }}\sum _{i=1}^{N}L[t_{\ast }^i-{\hat{w}}_{\ast }^T\Phi (P^i)]+\lambda R({\hat{w}}_{\ast })\end{array}$$

where $L$ is the loss function, $R$ is the regularization function.

The regression targets $t_{\ast }$ for the training pair $(P,G)$ are defined as
$$\begin{array}{rl}{t}_x& =(G_x-P_x)/P_w\\ t_y& =(G_y-P_y)/P_h\\ t_w& =log(G_w/P_w)\\ t_h& =log(G_h/P_h)\end{array}$$
It is easy to get
$$\begin{array}{rl}{\delta }_x& =(G_x-{\hat{G}}_x)/P_w\\ {\delta }_y& =(G_y-{\hat{G}}_y)/P_h\\ {\delta }_w& =log(G_w/{\hat{G}}_w)\\ {\delta }_h& =log(G_h/{\hat{G}}_h)\end{array}$$

…care must be taken when selecting which training pairs $(P,G)$ to use. Intuitively, if $P $ is far from all ground-truth boxes, then the task of transforming $P$ to a ground-truth box $G$ does not make sense.

Faster RCNN

BBox regression of RPN is a variant of Basic bbox regression.

Region Proposal Network (RPN)

This architecture is naturally implemented with an n×n convolutional layer followed by two sibling 1 × 1 convolutional layers (for reg and cls, respectively).

Translation-Invariant Anchors

Multi-Scale Anchors as Regression References

$P$ is equivalent to anchor box here, so anchors are proposal/references.

$\Phi (P)$ is 1 x 1 x C’ feature at the anchor position on the intermediate layer, and $w_{\ast }$ is a 1 x 1 x C’ convolution kernal.

SSD

Bbox regression in SSD is a simplified version of Faster RCNN bbox regression.

The main difference between them is:

1. SSD remove the intermediate layer and use 3x3 convolution in cls/reg layer. In MobileNet-SSD, use 1x1 convolution in cls/reg layer.
2. SSD only predict k scores for foreground in cls layer, background is also predicted but not use.

$P$ is called prior box or default box. but actually equivalent to anchor box.