【阅读笔记】Improved Training of Wasserstein GANs

Improved Training of Wasserstein GANs

Gulrajani I, Ahmed F, Arjovsky M, et al. Improved training of wasserstein gans[C]//Advances in Neural Information Processing Systems. 2017: 5767-5777.
GitHub: https://github.com/igul222/improved_wgan_training

Abstract

GAN虽然是个强有力的生成模型，但是训练不稳定的缺点影响它的使用。刚刚提出的 Wasserstein GAN (WGAN) 使得 GAN 的训练变得稳定，但是有时也会产生很差的样本和不收敛。我们发现这些问题的原因常常是因为 weight clipping 来满足 判别器（critic，os.坑，研究了半天才领会这个意思）的 Lipschitz constraint。我们把 weight clipping 转化为成 判别器 的梯度范数关于输入的惩罚。我们的方法优于 standard WGAN 和大部分的 GAN 的变种。

Introduction

Generative adversarial networks

Formally, the game between the generator G and the discriminator D is the minimax objective:
$$mi{n}_{G}ma{x}_D{E}_{x\sim p_r}[logD(x)]+E_{\hat{x}\sim p_g}[log(1-D(\hat{x}))]$$

In practice, the generator is instead trained to maximize $E_{\hat{x}\sim p_g}[log(D(\hat{x}))]$。因为这样可以规避当判别器饱和时的梯度消失。

Wasserstein GANs

The WGAN value function is constructed using the Kantorovich-Rubinstein duality to obtain
m i n G m a x D ∈ D E x ∼ p r [ D ( x ) ] − E x ^ ∼ p g [ D ( x ^ ) ] min_Gmax_{D\in\mathscr{D}}E_{x\sim p_r}[D(x)]-E_{\hat{x}\sim p_g}[D(\hat{x})] minG​maxD∈D​Ex∼pr​​[D(x)]−E