Weakness of adversarial training: overfit to the attack in use and hence does not generalize to test data
Curriculum adversarial training
思想:train model from weak attack to strong attack
方法
Let l l l denote the attack strength, K K K denote the maximal attack strength. A ( l ) \mathcal{A}(l) A(l) denotes an attack class parameterized with l l l.
Basic curriculum learning
i). start from no attack;
ii). train the model for one epoch and, once finished, calculate the l ~ \tilde{l} l~-accuracy;
iii-a). if l ~ \tilde{l} l~ increases at least once over the last 10 epoches, continue training;
iii-b). if l ~ \tilde{l} l~ does not increase over the last 10 epoches, set the parameters of the model to be the best ones (i.e. 10 epoches ago), and increase l l l by 1;
iv). Stop when