Curriculum adversarial training

Weakness of adversarial training: overfit to the attack in use and hence does not generalize to test data

Curriculum adversarial training

思想：train model from weak attack to strong attack

方法

Let $l$ denote the attack strength, $K$ denote the maximal attack strength. $\mathcal{A}(l)$ denotes an attack class parameterized with $l$.

Basic curriculum learning

i). start from no attack;
ii). train the model for one epoch and, once finished, calculate the $\tilde{l}$-accuracy;
iii-a). if $\tilde{l}$ increases at least once over the last 10 epoches, continue training;
iii-b). if $\tilde{l}$ does not increase over the last 10 epoches, set the parameters of the model to be the best ones (i.e. 10 epoches ago), and increase $l$ by 1;
iv). Stop when