TriggerBN ++

motivation

用两个BN(一个用于干净样本, 一个用于对抗样本), 结果当使用${\mathrm{B}\mathrm{N}}_{nat}$的时候, 精度能够上升, 而使用${\mathrm{B}\mathrm{N}}_{adv}$的时候, 也有相当的鲁棒性. 原文采用的是
$$\alpha \mathcal{L}(f(x),y)+(1-\alpha )\mathcal{L}(f(x+\delta ),y),$$
来训练(这里$f(x)$输出的是概率向量而非logits), 试试看别的组合方式, 比如
$$\mathcal{L}(\alpha f(x_{nat})+(1-\alpha )f(x_{adv}),y).$$

settings

Attribute	Value
attack	pgd-linf
batch_size	128
beta1	0.9
beta2	0.999
dataset	cifar10
description	AT=0.5=default-sgd-0.1=pgd-linf-0.0314-0.25-10=128=default
epochs	100
epsilon	0.03137254901960784
learning_policy	[50, 75] x 0.1
leverage	0.5
loss	cross_entropy
lr	0.1
model	resnet32
momentum	0.9
optimizer	sgd
progress	False
resume	False
seed	1
stats_log	False
steps	10
stepsize	0.25
transform	default
weight_decay	0.0005

results

x轴为$\alpha$从$0$变化到$1$.

	Accuracy	Robustness
$0.5{\mathcal{L}}_{nat}+0.5{\mathcal{L}}_{adv}$
$\mathcal{L}(0.5p_{nat}+0.5p_{adv},y)$
$0.1{\mathcal{L}}_{nat}+0.9{\mathcal{L}}_{adv}$ 48.350
$\mathcal{L}(0.1p_{nat}+0.9p_{adv},y)$ 48.270
$0.2{\mathcal{L}}_{nat}+0.8{\mathcal{L}}_{adv}$ 48.310
$\mathcal{L}(0.2p_{nat}+0.8p_{adv},y)$ 47.960

似乎原来的形式情况更好一点.