用各种GAN生成正态分布

用GAN生成正态分布

    毕设中有一部分与GAN（Generative Adversarial Networks）相关，但是一直不work，因此准备重新从最简单的GAN入手，实现一下试试看能不能发现什么问题。
    本文会用GAN从标准正态噪声生成均值为3.5，标准差为0.7的正态分布 $N(3.5,0.7^2)$。

---

整体结构

    一个GAN由生成器Generator（G）和判别器Discriminator（D）构成，D希望能正确分辨真实样本和G生成的假样本，G希望能骗过D，让D认为G生成的样本是真的。根据GAN的不同，G和D的目标函数有所不同，结构和连接方式也各有不同，G与D内部的结构也比较自由（姑且这么认为，实际上还是需要小心设计的）。
    下图是一个原始GAN的结构。$z$是一个$d$维的噪声向量，$x$是从目标分布 $N(3,0.5^2)$ 中采样得到的样本，即真实样本，也是一个$d$维向量。

---

生成器与判别器

    由于任务比较简单，只是随机向量到随机向量，所以G和D的结构只使用MLP。方便起见，令随机向量维度$d=10$。

# 生成器
class Generator_MLP(BasicBlock):
    def __init__(self, name=None):
        super(Generator_MLP, self).__init__(None, name or "Generator_MLP")
    def __call__(self, z, is_training=True, reuse=False):
        with tf.variable_scope(self.name, reuse=reuse):
            net = tf.nn.softplus(dense(z, 64, name='g_fc1'))
            out = dense(net, 10, name='g_fc2')
            return out
# 判别器
class Discriminator_MLP(BasicBlock):
    def __init__(self, name=None):
        super(Discriminator_MLP, self).__init__(None, name or "Discriminator_MLP")
    def __call__(self, x, is_training=True, reuse=False):
        with tf.variable_scope(self.name, reuse=reuse):
            net = tf.nn.tanh(dense(x, 64, name='d_fc1'))
            net = tf.nn.tanh(bn(dense(net, 64, name='d_fc2'), is_training, name='d_bn2'))
            yd = dense(net, 1, name="D_dense")
            return yd, net

---

目标函数

GAN

$$\begin{array}{c}{L}_D=E[log(D(x))]+E[log(1-D(G(z))]\\ \\ L_G=E[log(D(G(z)))]\end{array}\text{\hspace{1em}(1)}$$

WGAN

$$\begin{array}{c}{L}_D=E[D(x)]-E[D(G(z))]\\ \\ L_G=E[D(G(z))]\\ \\ W_D\leftarrow clip(W_D,-0.1,0.1)\end{array}\text{\hspace{1em}(2)}$$

LSGAN

$$\begin{array}{c}{L}_D=E[(D(x)-1{)}^2]+E[D(G(z{)}^2]\\ \\ L_G=E[(D(G(z))-1{)}^2]\end{array}\text{\hspace{1em}(1)}$$

代码实现

def build_placeholder(self):
        self.noise = tf.placeholder(shape=(self.batch_size, self.noise_dim), dtype=tf.float32)
        self.source = tf.placeholder(shape=(self.batch_size, self.noise_dim), dtype=tf.float32)

def build_gan(self):
    self.G = self.generator(self.noise, is_training=True, reuse=False)
    self.G_test = self.generator(self.noise, is_training=False, reuse=True)
    self.logit_real, self.net_real = self.discriminator(self.source, is_training=True, reuse=False)
    self.logit_fake, self.net_fake = self.discriminator(self.G, is_training=True, reuse=True)

def build_optimizer(self):
    if self.gan_type == 'gan':
        self.D_loss_real = tf.reduce_mean(tf.nn.sigmoid_cross_entropy_with_logits(logits=self.logit_real, labels=tf.ones_like(self.logit_real)))
        self.D_loss_fake = tf.reduce_mean(tf.nn.sigmoid_cross_entropy_with_logits(logits=self.logit_fake, labels=tf.zeros_like(self.logit_fake)))
        self.D_loss = self.D_loss_real + self.D_loss_fake
        self.G_loss = tf.reduce_mean(tf.nn.sigmoid_cross_entropy_with_logits(logits=self.logit_fake, labels=tf.ones_like(self.logit_fake)))
    elif self.gan_type == 'wgan':
        self.D_loss_real = - tf.reduce_mean(self.logit_real) 
        self.D_loss_fake = tf.reduce_mean(self.logit_fake)
        self.D_loss = self.D_loss_real + self.D_loss_fake
        self.G_loss = - self.D_loss_fake
        if self.clip_num:
            print "GC"
            self.D_clip = [v.assign(tf.clip_by_value(v, -self.clip_num, self.clip_num)) for v in self.discriminator.weights]
    elif self.gan_type == 'lsgan':
        def mse_loss(pred, data):
            return tf.sqrt(2 * tf.nn.l2_loss(pred - data)) / self.batch_size
        self.D_loss_real = tf.reduce_mean(mse_loss(self.logit_real, tf.ones_like(self.logit_real)))
        self.D_loss_fake = tf.reduce_mean(mse_loss(self.logit_fake, tf.zeros_like(self.logit_fake)))
        self.D_loss = 0.5 * (self.D_loss_real + self.D_loss_fake)
        self.G_loss = tf.reduce_mean(mse_loss(self.logit_fake, tf.ones_like(self.logit_fake)))
        if self.clip_num:
            print "GC"
            self.D_clip = [v.assign(tf.clip_by_value(v, -self.clip_num, self.clip_num)) for v in self.discriminator.weights]

---

结果

完整代码

https://github.com/SongDark/generate_normal