【论文笔记】Distribution Alignment: A Unified Framework for Long-tail Visual Recognition

摘要

we develop an adaptive calibration function that enables us to adjust the classification scores for each data point. We then introduce a generalized re-weight method in the two-stage learning to balance the class prior, which provides a flexible and unified solution to diverse scenarios in visual recognition tasks.

引言

一般的做法：
a deep network model has to simultaneously cope with imbalanced annotations among the head and medium-sized classes, and few-shot learning in the tail classes.
我们的工作：
Our approach focuses on improving the second-stage training of the classifier after learning a feature representation in a standard manner. To this end, we develop a unified distribution alignment strategy to calibrate the classifier output via matching it to a reference distribution of classes.
所提出方法的作用：
Such an alignment strategy enables us to exploit the class prior and data input in a pricipled manner for learning class decision boundary, which eliminates the needs for tedious hyper-parameter tuning and can be easily applied to various visual recognition tasks.
具体措施（分为两个components)
(1) in the first component, we introduce an adaptive calibration function that equips the class scores with an input-dependent, learnable magnitude and margin.
(2) in the second component, explicitly incorporates a balanced class prior by employing a generalized re-weight design for the reference class distribution, which provides a unified strategy to cope with diverse scenarios of label imbalance in different visual recognition tasks.

相关工作

One-stage Imbalance Learning(一阶段不平衡学习)
leverage the re-balancing ideas or explore knowledge transfer from head categories.
The basic idea of resample-based methods is to over-sample the minority categories or to under-sample the frequent categories in the training process.
Class-level methods re-weight the standard loss with category-specific coefficients correlated with the sample distributions.
Other work aim to enhance the representation or classifier head of tail categories by transferring knowledge from the head classes.
总的来说，这类方法的缺陷是只能针对特定网络模块设计的,泛化性不足.
These methods require designing a task specific network module or structure, which is usually non-trival to be generalized to different vision tasks.

Two-stage Imbalance Learning(二阶段不平衡学习)
Decouple proposes an instanced-balanced sampling scheme, which generates more generalizable representations and achieves strong performance after properly re-balancing the classifier heads.
这类方法的缺陷是很依赖超参数的调整
such a two-stage strategy typically relies on heuristic design in order to adjust the decision boundary of initially learned classifiers and requires tedious hyper-parameter tuning in practice.

方法

Two-stage learning framework
(1) learns a feature representation and a classifier head from the unbalanced data
(2) a calibration stage that adjusts the classification scores

Distribution Alignment

(1) Adaptive Calibration Function
在second stage中的classifier head, 我们提出这一策略来融合分类器O origin classifier head (frozen) and a learned class prior P in an input-dependent manner.
具体步骤:
<1> 获得分类器O的输出z
<2> 引入class-specific linear transform 来调整score
<3>构建confidence score function 来结合original and the transformed class scores
<4>再通过linear layer+sigmoid进行非线性映射
<5>softmax function 来校正

(2) Alignment with Generalized Re-weighting
denote the reference distribution P, we aim to minimize the expected KL-divergence between Pr and model prediction Pm.
其中Pr=w*a, a是Kronecker delta function ( equal 1 if y=c, otherwise equals 0 ). w=(1/rc)^p /(sum(1/rk))^p

有意思的探索(一)

which model compenents impose a bottleneck on its balanced performance.
本次实验分别采用instance-balanced, class-balanced, or square-root sampling三种进行对比. 首先利用上述采样方式进行训练.
其次,冻结编码部分,更换一个平衡类别的数据集来训练分类头.
结果显示, 第一阶段的表征学习都很强,并且instance-balanced可以达到最好的效果.这就说明分类器在不受到类别是否均衡的情况下,编码部分不会受到不平衡数据的影响.

Those empirical results indicate that the biased decision boundary in the feature space seems to be the performance bottleneck of the existing long-tail methods.

有意思的发现(二)

LWS and t-normalized try to achieve a similar goal by learning a magnitude scale and discarding the margin adjustment.
All these methods can be considered as the special cases of our DisAlign approach, which provides a unified and simple form to model the distribution mismatch in a learnable way.
Moreover, the resample based strategy is not easy to be applied for the instance-level or pixel-level tasks, our generalized re-weight provides an alternative solution to incorporate the class prior in a simple and effective manner.