WGAN-GP

WGAN不足

为了限制D为the set of 1-Lipschitz functions ，WGAN采用clip the weights of the critic to lie
within a compact space [−c, c]. 但这并未真正限制。

Difficulties with weight constraints
1.Capacity underuse
the critic trained with weight clipping ignores higher moments of the data dis-tribution and instead models very simple approximations to the optimal functions.
2.Exploding and vanishing gradients
interactions between the weight constraint and the cost function, which result in either vanishing or exploding gradients without careful tuning of the clipping threshold c.

WGAN-GP的Gradient penalty

Ppenalty（真正对判别器产生实质影响的区域）产生过程：
从Pdata中采样一个点，从PG中采样一个点
将这两个点连线
在连线之上在采样得到一个点，就是一个从penalty采样的一个点.
重复上面的过程就能不断采样得到x∼Ppenaltyx∼Ppenalty 。最终得到下图中的蓝色区域就可以看作是Ppenalty。

wgan-gp保留了WGAN loss衡量质量的特点，同时无需critic batch normalization。we penalize the norm of the critic’s gradient with respect
to each input independently, and not the entire batch. 建议使用layer normalization as a drop-in replacement for batch normalization
惩罚项为平方实现了Two-sided penalty。这个惩罚项的目的是让梯度尽可能趋向于等于1. 即当梯度大于1或小于1时都会受到惩罚。
参考https://blog.csdn.net/zh20166666/article/details/83444089