[DeFRCN] Decouple Faster R-CNN for Few-Shot Object Detection(ICCV 2021)

1. Motivation

​ 本文基于Transfer-Learning Based 以及 Faster R-CNN进行改进。

​ 本文针对分类和回归任务中存在的矛盾点进行分析：

• In this paper, we look closely into the conventional Faster R-CNN and analyze its contradictions from two orthogonal perspectives, namely multi-stage (RPN vs. RCNN) and multi-task (classification vs. localization).

2. Contribution

本文关于网络结构的2个创新点， Gradient Decoupled Layer用于多阶段的解耦，Prototypical Calibration Block用于多任务的解耦。

其中GDL是针对backbone，来解耦之前层和后面层；而PCB则是offline的prototype的分类层，用于boost原有的分类层。

• We look closely into the conventional Faster R-CNN and propose a simple yet effective architecture for few-shot detection, named Decoupled Faster R-CNN, which can be learned end-to-end via straightforward fine-tuning.
• To deal with the data-scarce scenario, we further present two novel modules, i.e. GDL and PCB, to perform de- coupling among multiple components of Faster R-CNN and boost classification performance respectively.
• DeFRCN is remarkably superior to SOTAs on various benchmarks, revealing the effectiveness of our approach.

3. Method

Backbone、RPN、Box Classifier以及Regressor在fine-tune阶段是trainable，而RCNN是frozen的。

Problem of multi-task learning

作者认为对于多任务学习来说，子网络的优化目标存在不一致性。

RPN是where to look, RCNN是what to look

classification head需要translation invariant features，而localization head 需要translation covariant features。

因此，可能导致一个suboptimal solution

Problem of shared backbone

由于Backbone的梯度回传和RCNN以及 RPN有关，但是这2者有一定的矛盾性，因此，作者认为这可能会导致FSOD性能的下降。并且在FSOD中，第二阶段的RPN会受到更多的前景-背景的混淆问题foreground-background confusion。因此可能造成对于base classes过拟合梯度的传播，到backbone以及RCNN

• which means a pro- posal that belongs to background in the base training phase is likely to be foreground in the novel fine-tuning phase

3.1 Gradient Decoupled Layer

• Perform Decoupling with GDL

  

• Optimization with GDL
  

3.2 Prototypical Calibration Blo