Trials-BBN

motivation

BBN 对于处理长尾问题非常有效, 我在想, 能不能类似地用在鲁棒问题上.

思想很简单, 就是上面用干净数据, 下面用对抗样本(其用$\alpha =0.5$的eval mode 生成), 但是结果非常差.

settings

- batch_size: 128

- beta1: 0.9

- beta2: 0.999

- dataset: cifar10

- epochs: 200

- epsilon: 0.03137254901960784

- eva_alpha: 0.5

- learning_policy: AT

- loss: cross_entropy

- lr: 0.1

- model: resnet32

- momentum: 0.9

- norm_cls: True | False

- optimizer: sgd

- progress: False

- resume: False

- seed: 1

- steps: 10

- stepsize: 0.25

- transform: default

- weight_decay: 0.0002

results

	Loss	Accuracy	Robustness
parabolic decay; norm_cls=False
fixed_alpha=0.5; norm_cls=False
parabolic decay; norm_cls=True
fixed_alpha=0.5; norm_cls=True

可以看得出来, 强行糅合在一起效果不好, 显然干净的features或者对抗的features占主导的时候精度能上去.