Domain Adaptive Faster R-CNN for Object Detection in the Wild（论文解读）

Abstract

We build our approach based on the recent state-of-the-art Faster R-CNN model, and design two domain adaptation components, on image level and instance level, to reduce the domain discrepancy. The two domain adaptation components are based on H-divergence theory, and are implemented by learning a domain classifier in adversarial training manner.
The domain classifiers on different levels are further reinforced with a consistency regularization to learn a domain-invariant region proposal network (RPN) in the Faster R-CNN model

• 什么是H-divergence theory

• 什么是 consistency regularization

4. Domain Adaptation for Object Detection

4.1. A Probabilistic Perspective

• 1. Image-Level Adaptation:

• 2. Instance-Level Adaptation

• 3. Joint Adaptation

4.2. Domain Adaptation Components

- 1.Image-Level Adaptation:

Since the receptive field of each activation corresponds to an image patch of the input image Ii, the domain classifier actually predicts the domain label for each image patch.

The benefits of this choice are twofold:

1. aligning image-level representations generally helps to reduce the shift caused by the global image difference such as image style, image scale, illumination, etc. A similar patch-based loss **has shown to be effective in recent work on style transfer **[29], which also deals with the global transformation,
2. the batch size is usually very small for training an object detection network, due to the use of high-resolution input. This patch-based design is helpful to increase the number of training samples for training the domain classifier.

i表示第i张图片，（u，v)是bottom feature map 经过image level分类器最后得到的feature map上的坐标。D表示domain类别。（点（u，v)输出其实就是这点的感受野的类别，即原图上一个patch的类别）
通过GRL层达到梯度反转，最终域间难以区分。

- 2.Instance-Level Adaptation

The instance-level representation** refers to the ROI-based feature vectors **before feeding into the final category classifiers (i.e., the rectangles after the “FC” layer)
Similar to the image-level adaptation, we train a domain classifier for the feature vectors to align the instance-level distribution.

i表示第i张图片，i,j表示第i张图片第个region proposal。其实Instance-Level分类器输入就是ROI之后的第二个FC层的输出向量（不明白为什么画成那样）
同样，通过GRL层达到梯度反转，最终域间难以区分。

- 3.Consistency Regularization

保证了第j个proposal的输出与图像层面的平均输出尽量相等。

4.3. Network Overview

6. Conclusion

The adaptation components are based on adversarial training ofH-divergence. A consistency regularizer is further applied to learn a domain-invariant RPN

问题

Each batch is composed of 2 images, one from the source domain and one from the target domain.
但是论文并没有提及如何训练。

猜测：对于faster rcnn loss只对源域，对于Image-Level Adaptation，loss基于俩张图片的输出，对于Instance-Level Adaptation（问题是对目标域如何生成proposal）
具体实现需要看代码，总的来说，end-to-end训练原理上是可以实现的。

对于我们的数据，生成proposal似乎是更容易的一件事，暂时没有啥想法。

思考：faster rcnn +域适应 用于癌细胞检测

源域：必须有bounding box(或者易得到bounding box)
目标域：我们目标域是带弱标注的，因此应当是弱监督方式的域适应，而不是无监督。
（对于目标域的损失函数要基于弱标注，例如，对于anchor中心离标注近的设置正样本？？？）

另外一个想法：使用在12年数据集上训练好的模型对16年数据检测，根据一定策略选择bounding box，从而制作数据集。用这个数据集去微调之前训练好的模型。

问题：2012数据集过小，作为源域是否合适？？？ 域适应方法一般来说，目标域数据不会大于源域。

2018cvpr有一篇论文 : 《Cross-Domain Weakly-Supervised Object Detection through Progressive Domain Adaptation》

2019_CVPR《Few-shot Adaptive Faster R-CNN》
2019_CVPR《Diversify and Match：A Domain Adaptive Representation Learning Paradigm for Object Detection》