[论文阅读 2020 CVPR 目标跟踪]SiamCAR: Siamese Fully Convolutional Classification and Regression for Visual

简介

paper:SiamCAR: Siamese Fully Convolutional Classification and Regression for Visual Tracking

code:ohhhyeahhh/SiamCAR

这篇论文提出的动机是：SiamRPN系列的跟踪器依赖于RPN来进行Classfication和Regression,而这些基于RPN的跟踪需要设置好anchor boxes相关参数才能达到比较好的跟踪效果，对调参的要求较高。基于此，这篇论文基于一个简单的网络结构，实现了一个简单高效的跟踪模型。

主要内容

上图是SiamCAR的网络结构，SiamCAR分为用于提取特征的Siamese Subnetwork和用于分类和回归的Classification-Regression Subnetwork两部分，而SiamCAR最重要的贡献点在于后半部分的Classification-regression Subnetwork.

Feature Extraction

SiamCAR的特征提取网络使用的是SiamRPN++中修改过的Resnet50,同SiamRPN++一样，采用了REsnet50后三层的特征经过cat后分别得到Template patch的feature map$\phi (Z)$和Search region的feature map$\phi (X)$.

之后采用SiamRPN++中的Depthwise Cross Correlation计算Correlation:

$$R=\phi (X)\star \phi (Z)$$

由于此时得到的response map R的通道数太大($256\ast 3$),所以作者使用了一个$1X1$的卷积层对response map进行了降维,减少了模型参数量并提高了模型推导速度。

Bounding Box Prediction

Classification-Regresssion Subnetwork又分为classification branch和regression branch.其中对于regression branch输出$(l,t,r,b)$，表示从相应位置到搜索区域中边界框四个边角的距离.而classification branch除了输出clasification，作者还添加了一个centerness branch,且centerness branch输出centerness score(动机是：作者观察到距离目标越远的bbox质量越低)

An observation is that the locations far away from the center of an target tend to produce low-quality predicted bounding boxes, which reduces the performance of the tracking system.

最终，对于regression branch的输出使用IOU loss，classification branch使用cross-entropy loss,而对于centerness branch使用如下损失:

$$C(i,j)=\mathbb{I}\left({\tilde{t}}_{(i,j)}\right)\ast \sqrt{\frac{\min (\tilde{l},\tilde{r})}{\max (\tilde{l},\tilde{r})}\times \frac{\min (\tilde{t},\tilde{b})}{\max (\tilde{t},\tilde{b})}}$$

$$\begin{array}{rl}{\mathcal{L}}_{cen}& =\frac{-1}{\sum \mathbb{I}\left({\tilde{t}}_{(i,j)}\right)}\sum _{\mathbb{I}\left({\tilde{t}}_{(i,j)}\right)==1}C(i,j)\ast \log A_{w\times h\times 1}^{cen}(i,j)+(1-C(i,j))\ast \log \left(1-A_{w\times h\times 1}^{cen}(i,j)\right)\end{array}$$

最终总损失为:

$$\mathcal{L}={\mathcal{L}}_{cls}+{\lambda }_1{\mathcal{L}}_{cen}+{\lambda }_2{\mathcal{L}}_{reg}$$

The Tracking Phase

最终跟踪时，对于位置$(i,j)$,网络会输出一个6维的向量$(cls,cen,l,t,r,b)$

For a location (i, j), the proposed method produces a 6D vector $T_{ij}=(cls,cen,l,t,r,b)$,where $cls$ represents the foreground score of classification, $cen$ represents the centerness socre, and $l+r$ and $t+b$ represent the predicted width and height of the target in current frame.

所以根据下面的式子计算highesdt score所在的位置$q$:

q = arg ⁡ max ⁡ i , j { ( 1 − λ d ) c l s i j × p i j + λ d H i j } q=\arg \max _{i, j}\left\{\left(1-\lambda_{d}\right) c l s_{i j} \times p_{i j}+\lambda_{d} H_{i j}\right\} q=argi,jmax​{(1−λd​)clsij​×pij​+λd​Hij​}

where $H$ is the cosine window and ${\lambda }_d$ is the balance weight. The output $q$ is a queried location with the highest score being a target pixel.

由于在$q$周围的都有可能是是trage pixel，所以作者又计算$q$的$n$个neightborhoods的$cl{s}_{ij}\ast p_{i,j}$得分，并取top-k的regression boxes进行加权平均得到最终的target box.

We observed that the pixels located around $q$ are more likely to be the target pixel. Hence we choose the top-k points from n neighborhoods of $q$ according to the value $cl{s}_{ij}\times p_{ij}$ . The final prediction is the weighted average of the selected $k$ regression boxes

实验结果

小结

SiamCAR用简单的模型就实现了SiamRPN++的性能，甚至要更好，这是非常不容易的,这个模型非常值得深入学习，就是不知道模型训练的难度如何；另外，作者的论文和代码都写的极好理解，给作者点个攒!