论文阅读 [CVPR-2022] BatchFormer: Learning to Explore Sample Relationships for Robust Representation Lea

论文阅读 [CVPR-2022] BatchFormer: Learning to Explore Sample Relationships for Robust Representation Learning

studyai.com

搜索论文: [Autoregressive Image Generation using Residual Quantization](http://www.studyai.com/search/whole-site/?q=Autoregressive Image+Generation+using+Residual+Quantization)

http://www.studyai.com/search/whole-site/?q=BatchFormer+Learning+to+Explore+Sample+Relationships+for+Robust+Representation+Learning

摘要(Abstract)

Despite the success of deep neural networks, there are still many challenges in deep representation learning due to the data scarcity issues such as data imbalance, unseen distribution, and domain shift.

尽管深度神经网络取得了成功，但由于数据不平衡、未见分布和领域转移等数据稀缺问题，深度表征学习仍然存在许多挑战。

To address the above mentioned issues, a variety of methods have been devised to explore the sample relationships in a vanilla way (i.e., from the perspectives of either the input or the loss function), failing to explore the internal structure of deep neural networks for learning with sample relationships.

为了解决上述问题，人们设计了多种方法，以虚化的方式（即从输入或损失函数的角度）探索样本关系，未能探索深度神经网络的内部结构，以进行样本关系学习。

Inspired by this, we propose to enable deep neural networks themselves with the ability to learn the sample relationships from each minibatch.

受此启发，我们建议使深层神经网络本身能够从每个小批量中学习样本关系。

Specifically, we introduce a batch transformer module or BatchFormer, which is then applied into the batch dimension of each mini-batch to implicitly explore sample relationships during training.

具体来说，我们引入了一个batch transformer模块或BatchFormer，然后将其应用到每个小批次的批次维度中，以隐式地探索训练期间的样本关系。

By doing this, the proposed method enables the collaboration of different samples, e.g., the head-class samples can also contribute to the learning of the tail classes for long-tailed recognition.

通过这样做，所提出的方法可以实现不同样本的协作，例如，头类样本也可以有助于长尾识别的尾类学习。

Furthermore, to mitigate the gap between training and testing, we share the classifier between with or without the BatchFormer during training, which can thus be removed during testing.

此外，为了减少训练和测试之间的差距，我们在训练期间使用或不使用批处理器的情况下共享分类器，因此可以在测试期间移除批处理器。

We perform extensive experiments on over ten datasets and the proposed method achieves significant improvements on different data scarcity applications without any bells and whistles, including the tasks of long-tailed recognition, compositional zero-shot learning, domain generalization, and contrastive learning.

我们在10多个数据集上进行了大量的实验，所提出的方法在不同的数据稀缺性应用上取得了显著的改进，没有任何细节，包括长尾识别、组合零拍学习、领域泛化和对比学习等任务。

Code will be made publicly available at https://github.com/zhihou7/BatchFormer.