D_pre

一、构建D和G

D是判别网络，有两个输入：希望判别fake输出为0，real输出为1.
G（x）,x表示噪音的数据，噪音的数据先通过G的网络进行生成，然后放入D，让判别网络来判别生成的数据为真还是假。x^表示真实输入为real。
G为生成网络，首先由一个噪音的输入，没有任何规则，希望G网络通过一系列的参数，比如说θ，将x生成为G(x)，G(x)能够与real数据越接近越好

二、D_pre

D网络判别的效果的好坏最终能够影响生成网络G，为了能使生成效果更好，需要将D网络做的更好一些。
网络都有权重参数wb，这些权重参数的初始化非常关键，对于D网络，不进行随机的初始化，而是进行预先的训练D_pre，D网络先进行一个判断，知道哪些是0哪些是1数据。

三、先构造一个model，定义一系列网络架构

def main(args):
    # 初始化GAN实例
    model = GAN(
        # 产生真实值
        DataDistribution(),
        # 产生生成值
        GeneratorDistribution(range=8),
        args.num_steps,                 #一共迭代1200次
        args.batch_size,                #一次迭代12个点的数据
        args.log_every,                 #隔多少次打印一次当前loss
    )
    model.train()

1、真实数据

class DataDistribution(object):
    def __init__(self):
        self.mu = 4             #均值
        self.sigma = 0.5        #标准差(初始状态蓝色的线)

    def sample(self, N):
        samples = np.random.normal(self.mu, self.sigma, N)
        samples.sort()
        return samples

2、生成数据

当我们用G网络的时候，我们需要先造一些噪音点，通过GeneratorDistribution随机的初始化一种分布，
将这种分布当成G网络的输入

class GeneratorDistribution(object):
    def __init__(self, range):  
        self.range = range      #随机的初始化分布(G网络的输入)

    def sample(self, N):
        return np.linspace(-self.range, self.range, N) + \
            np.random.random(N) * 0.01

3、定义好参数之后构造GAN模型

class GAN(object):
    # 初始化变量
    def __init__(self, data, gen, num_steps, batch_size, log_every):
        self.data = data
        self.gen = gen
        self.num_steps = num_steps
        self.batch_size = batch_size
        self.log_every = log_every
        self.mlp_hidden_size = 4            #神经网络非常简易，只有4个神经元
        
        self.learning_rate = 0.03

        self._create_model()

4、_create_model

 def _create_model(self):
        # 建立预判别模型
        with tf.variable_scope('D_pre'):#在D_pre域当中构造D_pre网络，为了训练获得D网络初始化参数
            self.pre_input = tf.placeholder(tf.float32, shape=(self.batch_size, 1))#pre_input.shape=[12,1]12是batch，1是1维的点
            self.pre_labels = tf.placeholder(tf.float32, shape=(self.batch_size, 1))#pre_labels.shape=[12,1]
            # 获得预测结果
            D_pre = discriminator(self.pre_input, self.mlp_hidden_size)#初始化
            # 预测值与真实之间的差异
            self.pre_loss = tf.reduce_mean(tf.square(D_pre - self.pre_labels))
            # 训练缩小预测值与真实值的差异
            self.pre_opt = optimizer(self.pre_loss, None, self.learning_rate)

        # This defines the generator network - it takes samples from a noise
        # distribution as input, and passes them through an MLP.
        # 建立生成模型
        with tf.variable_scope('Gen'):
            # 伪造数据的生成
            self.z = tf.placeholder(tf.float32, shape=(self.batch_size, 1))
            self.G = generator(self.z, self.mlp_hidden_size)

        # The discriminator tries to tell the difference between samples from the
        # true data distribution (self.x) and the generated samples (self.z).
        # Here we create two copies of the discriminator network (that share parameters),
        # as you cannot use the same network with different inputs in TensorFlow.
        # 建立判别模型
        with tf.variable_scope('Disc') as scope:
            # 对真实值做预测， D1为真实值的概率
            self.x = tf.placeholder(tf.float32, shape=(self.batch_size, 1))
            self.D1 = discriminator(self.x, self.mlp_hidden_size)
            # 变量重用
            scope.reuse_variables()
            # 对造假值做预测， D2为预测到造假值的概率
            self.D2 = discriminator(self.G, self.mlp_hidden_size)

        # Define the loss for discriminator and generator networks (see the original
        # paper for details), and create optimizers for both
        self.loss_d = tf.reduce_mean(-tf.log(self.D1) - tf.log(1 - self.D2))
        self.loss_g = tf.reduce_mean(-tf.log(self.D2))

        self.d_pre_params = tf.get_collection(tf.GraphKeys.TRAINABLE_VARIABLES, scope='D_pre')
        self.d_params = tf.get_collection(tf.GraphKeys.TRAINABLE_VARIABLES, scope='Disc')
        self.g_params = tf.get_collection(tf.GraphKeys.TRAINABLE_VARIABLES, scope='Gen')
        # 获得训练以后的参数
        self.opt_d = optimizer(self.loss_d, self.d_params, self.learning_rate)
        self.opt_g = optimizer(self.loss_g, self.g_params, self.learning_rate)

i、D_pre = discriminator(self.pre_input, self.mlp_hidden_size)进行初始化操作

def discriminator(input, h_dim):#input就是需要判别的数据，真实?生成?h_dim * 2隐藏神经元个数'd0'指定命名域linear控制w参数和b参数初始化
    h0 = tf.tanh(linear(input, h_dim * 2, 'd0'))
    h1 = tf.tanh(linear(h0, h_dim * 2, 'd1'))   
    h2 = tf.tanh(linear(h1, h_dim * 2, scope='d2'))

    h3 = tf.sigmoid(linear(h2, 1, scope='d3'))
    return h3

通过一个linear的函数来控制w参数和b参数的初始化

input大小是等于12x1的，刚刚定义的隐层有8个神经元，所以w是1x8的，b的shape为8

#定义linear， 用于进行卷积
def linear(input, output_dim, scope=None, stddev=1.0):
    norm = tf.random_normal_initializer(stddev=stddev)  #w参数随机的初始化
    const = tf.constant_initializer(0.0)                #b参数直接初始化为常量0 
    with tf.variable_scope(scope or 'linear'):
        w = tf.get_variable('w', [input.get_shape()[1], output_dim], initializer=norm)#声明w的shape大小
        b = tf.get_variable('b', [output_dim], initializer=const)
        return tf.matmul(input, w) + b

ii、完成了D_pre，定义loss

            # 获得预测结果
            D_pre = discriminator(self.pre_input, self.mlp_hidden_size)#初始化
            # 预测值与真实之间的差异
            self.pre_loss = tf.reduce_mean(tf.square(D_pre - self.pre_labels))
            # 定义优化器进行求解
            self.pre_opt = optimizer(self.pre_loss, None, self.learning_rate)

定义optimizer函数，用于优化参数，学习率不断衰减的学习策略

def optimizer(loss, var_list, initial_learning_rate):
    # 学习率衰减系数
    decay = 0.95                                    #每次衰减为原来的0.95
    num_decay_steps = 150                           #每迭代150次进行一次学习率衰减
    batch = tf.Variable(0)
    #进行学习率的衰减
    learning_rate = tf.train.exponential_decay(     #tf的学习率衰减的学习方式
        initial_learning_rate,
        batch,
        num_decay_steps,
        decay,
        staircase=True
    )
    # 梯度下降求解器
    optimizer = tf.train.GradientDescentOptimizer(learning_rate).minimize(
        loss,
        global_step=batch,
        var_list=var_list
    )
    return optimizer

至此完成了D_pre预先训练的网络模型，目的是想拿出一组差不多的w参数和b参数来初始化真正的判别网络