[Retentive R-CNN] Generalized Few-Shot Object Detection without Forgetting(CVPR. 2021)

1. Motivation

本文关注于fine-tune后的FSOD模型会在base classes上性能下降的问题。这篇文章构建了Retentive R-CNN，创新点在于Bias-Balance RPN Re-detector，用来在识别novel classes的同时，不降低原有的base classes的精度

• However, the majority focus merely on the performance of few-shot categories and ignore the catastrophic forgetting of base classes, which is not realistic.

Meta-learning方法的缺陷：

​ 由于使用support images，那么如果当support category较多的情况下，那么网络训练的时间复杂度也会增加

• As their computational complexity is proportional to the number of categories, these methods become rather slow or even unavailable when tackling both sets of classes of a dataset

本文还划分了目前FSOD的主要方法，可以分为Meta Learning Based以及 Transfer-learning Baed

• Meta Learning Based

  FSRW、 Meta R-CNN、 Few-Shot Object Detection with Attention-RPN and Multi-Relation Detector、Repmet、 MetaDet

• Transfer Learning Based

  LSTD、TFA、MPSR、Context-Transformer

2. Contribution

• Our contributions can be concluded as follows:
  • We find properties of base class detectors neglected in few-shot detection literature, which can be utilized to improve both base and novel class performance for transfer learning based methods with little overhead.
  • We propose a few-shot detector without forgetting, Retentive R-CNN, with Bias-Balanced RPN and Re-detector to assist novel class adaptation with base class knowledge and ensemble base and novel class detectors.
  • Our method achieves state-of-the-art overall performance on the few-shot detection benchmark[41, 17] across all settings, with leading base class metrics and competitive novel class metrics.

3. Method

3.1 Analysis on Transfer Learning based Few-Shot Object Detection

作者在TFA的基础上，做了以下3个实验 。

3.1.1 Why cosine classifier works?

图2（a）所示，通过L2正则化可以发现novel classs 和base class 被区分开来，并且novel class 和base class 相关性强的，L2正则化后的得分也会比较高。

• The results are shown in Figure2(a). A massive variation of norms between base classes and unseen novel classes can be easily observed.
• Also, the norms of unseen classes with closer relationship with seen classes are relatively higher (blue names annotated in Figure2(a))

3.1.2 Does b