用各种GAN生成MNIST数字
完整代码:https://github.com/SongDark/GAN_collections
数据获取
MNIST数据集的获取可以参考这篇博客:
https://blog.csdn.net/songbinxu/article/details/82992264
需要注意的是MNIST数据的值域范围,有的源是原本的
[
0
,
255
]
[0,255]
[0,255],有的归一化到了
[
0
,
1
]
[0,1]
[0,1]。
生成器和判别器
实现了CNN和MLP两种版本。CNN结构参考自这里,MLP结构参考自这里。
有关 Spectral Norm
在dense
层、conv2d
层和deconv2d
层中都实现了Spectral Normalization,可以自由选择或取消。Spectral Normalization是用来限制判别器D的,不应该加到生成器G中。
有关 Gradient Clipping
所谓 Gradient Clipping(GC) 就是将参数限制在某个范围内,比如 ( − 0.01 , 0.01 ) (-0.01,0.01) (−0.01,0.01),它也是用来限制判别器D的,不应该加到生成器G中。WGAN的原始版本必须加GC,以保证判别器D满足Lipschiz连续条件。我的经验是,即使是其他GAN类型,有时候加上GC也有奇效,比如在LSGAN里,没有加GC时是无法生成MNIST的,加上之后就好了。
各种GAN的目标函数
目前涉及到的有GAN、WGAN、WGAN-GP、DRAGAN、LSGAN和CGAN。
以后有空再补其他的。
GAN
(1) L D = E [ l o g ( D ( x ) ) ] + E [ l o g ( 1 − D ( G ( z ) ) ] L G = E [ l o g ( D ( G ( z ) ) ) ] \begin{matrix} L_D=E[log(D(x))] + E[log(1-D(G(z))] \\ \\ L_G=E[log(D(G(z)))] \\ \end{matrix} \tag{1} LD=E[log(D(x))]+E[log(1−D(G(z))]LG=E[log(D(G(z)))](1)
WGAN
(2) L D = E [ D ( x ) ] − E [ D ( G ( z ) ) ] L G = E [ D ( G ( z ) ) ] W D ← c l i p ( W D , − 0.1 , 0.1 ) \begin{matrix} L_D=E[D(x)]-E[D(G(z))] \\ \\ L_G=E[D(G(z))] \\ \\ W_D\leftarrow clip(W_D,-0.1,0.1) \end{matrix} \tag{2} LD=E[D(x)]−E[D(G(z))]LG=E[D(G(z))]WD←clip(WD,−0.1,0.1)(2)
WGAN-GP
(3) L D = L D W G A N + λ E [ ( ∣ ▽ D ( α x − ( 1 − α G ( z ) ) ) ∣ − 1 ) 2 ] L G = L G W G A N \begin{matrix} L_D=L_D^{WGAN}+\lambda E[(|\bigtriangledown D(\alpha x-(1-\alpha G(z)))| - 1)^2] \\ \\ L_G=L_G^{WGAN}\\ \end{matrix} \tag{3} LD=LDWGAN+λE[(∣▽D(αx−(1−αG(z)))∣−1)2]LG=LGWGAN(3)
LSGAN
(4) L D = E [ ( D ( x ) − 1 ) 2 ] + E [ D ( G ( z ) 2 ] L G = E [ ( D ( G ( z ) ) − 1 ) 2 ] \begin{matrix} L_D=E[(D(x)-1)^2] + E[D(G(z)^2] \\ \\ L_G=E[(D(G(z))-1)^2] \\ \end{matrix} \tag{4} LD=E[(D(x)−1)2]+E[D(G(z)2]LG=E[(D(G(z))−1)2](4)
DRAGAN
(5) L D = L D G A N + λ E [ ( ∣ ▽ D ( α x − ( 1 − α x p ) ) ∣ − 1 ) 2 ] L G = L G G A N \begin{matrix} L_D=L_D^{GAN}+\lambda E[(|\bigtriangledown D(\alpha x-(1-\alpha x_p))| - 1)^2] \\ \\ L_G=L_G^{GAN}\\ \end{matrix} \tag{5} LD=LDGAN+λE[(∣▽D(αx−(1−αxp))∣−1)2]LG=LGGAN(5)
CGAN
(6) L D = E [ l o g ( D ( x , c ) ) ] + E [ l o g ( 1 − D ( G ( z ) , c ) ] L G = E [ l o g ( D ( G ( z ) , c ) ) ] \begin{matrix} L_D=E[log(D(x,c))] + E[log(1-D(G(z),c)] \\ \\ L_G=E[log(D(G(z),c))] \\ \end{matrix} \tag{6} LD=E[log(D(x,c))]+E[log(1−D(G(z),c)]LG=E[log(D(G(z),c))](6)