learning to reweight examples for robust deep learning

deep neural networks can easily overfit to training biases and label noises. In addition to various regularizers, example reweighting algorithms are popular solutions.

They propose a novel meta-learning algorithm that learns to assign weights to training examples based on their gradient directions. And they propose an online reweighting method that leverages an additional small validation set and adaptively assigns importance weights to examples in every iteration. Basiclly, their method improves the training objective through a weighted loss rather than an average loss and is an instantiation of meta-learning, i.t. learning to learn better. 

To determine the example weights, their method performs a meta gradient descent step on the current mini-batch example weights to minimize the loss on a clean unbiased validation set.

Training set bias includes class imbalance and label noise. (others, e.g., dataset adversarial attack)

In noisy label problems, we prefer examples with smaller training losses as they are more likely to be clean images; yet in class imbalance problems, algorithms such as hard negative mining prioritize examples with higher training loss since they are more likely to be the minority class.

Solving the training set bias problems is inherently ill-defined, and stronger regularization can usually work surprisingly well in certain synthetic noise settings. 

The authors argue that in order to learn general forms of training set biases, it is necessary to have a small unbiased validation to guide training. 

Different from existing traing loss based approaches, they follow a meta-learning paradigm and model the most basic assumption: the best example weighting should minimize the loss of a set of unbiased clean validation examples that  are consistent with the evaluation procedure. 

Hard examples are not always preferred in the process of outliers and noise processes. In self-paced learning, example weights are obtained through optimizing the weighted training loss encouraging learning easier examples first. In each step, the learning algorithm jointly solves a mixed integer program that iterates optimizing over model parameters and binary example weights.?

In addition to corrupted data, Koh & Liang (2017); Munoz-Gonzalez et al. (2017) demonstrate the possibility of a dataset adversarial attack (i.e. dataset poisoning). Using validation loss as the meta-objective has been explored in recent meta-learning literature for few-shot learning (Ravi & Larochelle, 2017; Ren et al., 2018; Lorraine & Duvenaud, 2018), where only a handful of examples are available for each class. 

确定了神经网络的模型，和损失函数，对于训练集和验证集都计算加权的损失：

带星号表示是最优的参数，用（2）得出来的最优的w去获得（1）中最优的θ，用（1）得出来的最优的 θ 去获得（2）中最优的w。

想要估计训练样本在validation set上的小的扰动(perturb)产生的影响，训练样本上参数的更新：

 那么寻找一个在validation set 上损失最优的扰动参数，对θ进行更新即可。然而这个过程是time-consuming的，采用下面的方式去估计参数w,并且进行了batch normalization:

 作者通过举例MLP，来演示怎样计算权重参数w。

Zl-1表示第l-1层经过激活函数之后的输出，θl 表示第l层的参数，所以：

 同时，z的梯度和θ的梯度表示给出来：

（12）式的证明重点是θt+1 对  ε 求导，参考式子：

 

（5）式左侧对 ε 求导，右侧θt 是常数，ε的系数是 fi对θ求导，所以下图红色方框部分可推得出来：

 第一个点积计算的是两个样本之间的相似度，第二个计算的是训练和验证样本的梯度的方向相似度。如果训练和验证样本很相似，方向也相似，那么损失f对ε求导越小，那么对应的权重越大越好。相反的情况，权重越小越好。

 在算法中，进行了两次forward， 三次backward，比起来普通的训练方式，多出来了3倍的训练时间。

同时作者证明了这样的重加权方式是收敛的。

证明和实验过程省略。