VAE（变分自编码器）

创造性人工智能应用—VAE变分自编码器（variational autoencoder)

1、从图像的潜在空间中采样

1、VAE（变分自编码器）：不是将图像压缩在潜在空间中的固定编码，而是将图像转换为统计分布的参数——平均值和方差

2、VAE使用平均值和方差，从分布中随机采样一个元素，并将这个元素解码到原始输入

3、随机过程：提高了稳健性，并迫使潜在空间中的任何位置都有其对应的意义表示（即潜在空间中采样的每个点都能解码为有效输出）

4、（1）编码器模块将输入样本中input_img转换为表示潜在空间的两个参数z_mean和z_log_variance

  (2)假定潜在分布能够生成输入图像，并从这个分布中随机采样一个点
  
  z_mean, z_log_variance = encoder(input_img) # 将输入编码为平均值和方差两个参数
  z = z_mean +exp(z_log_variance) * eplison # 使用小随机数抽取一个潜在点
  resconstructed_img = decoder(z) # 将z解码为一张图片
  model = Model(input_img, resconstructed_img)

5、连续性和潜在空间的低维度：使得潜在空间的每个方向都表示数据中的有意义的变化轴

# coding: utf-8
from keras import layers
from keras import backend as K
from keras.models import Model
from keras import Input
import numpy as np


img_shape = (28, 28, 1)
batch_size = 16
latent_dim = 2 # 潜在空间维度二维平面

# -------------------利用函数式API构建VAE网络---------------------------------------------
input_img = Input(shape=img_shape)
x = layers.Conv2D(32, 3, padding='same', activation='relu')(input_img) # 利用padding=‘same’模型的输入层和输出层的维度一样
x = layers.Conv2D(64, 3, padding='same', activation='relu', strides=(2, 2))(x)
x = layers.Conv2D(64, 3, padding='same', activation='relu')(x)
x = layers.Conv2D(64, 3, padding='same', activation='relu')(x)

shape_before_flattening = K.int_shape(x) # 返回张量尺寸

x = layers.Flatten()(x)
x = layers.Dense(32, activation='relu')(x)

# 输入图像被编码为均值和方差
z_mean = layers.Dense(latent_dim)(x)
z_log_variance = layers.Dense(latent_dim)(x)

print(shape_before_flattening)

潜在空间采样的函数
(在Keras中，任何对象都是层，如果没有内置的函数，就要将代码包装到一个Lambda层中，即自定义层)
args（参数）

def sampling(args):
    z_mean, z_log_variance = args
    eplison = K.random_normal(shape=(K.shape(z_mean)[0], latent_dim), mean=0, stddev=1.)
    
    return z_mean + K.exp(z_log_variance) * eplison

z = layers.Lambda(sampling)([z_mean, z_log_variance])

解码器的实现：
将向量Z的尺寸调整为图像的大小尺寸，使用几个卷积层来得到最终的图像输出，它和原始的图像input_img相同大小

decoder_input = layers.Input(K.int_shape(z)[1:])

# 对输入进行上采样
x = layers.Dense(np.prod(shape_before_flattening[1:]), activation='relu')(decoder_input)

# 将z转化为特征图，使其形状与编码器模型最后一个Flatten层之前的特获图形状一致
x= layers.Reshape(shape_before_flattening[1:])(x)

# 将z解码为与原始输入图像具有相同尺寸的特征图
x = layers.Conv2DTranspose(32, 3, padding='same', activation='relu', strides=(2, 2))(x)

x = layers.Conv2D(1, 3, padding='same', activation='sigmoid')(x)

decoder = Model(decoder_input, x)
decoder.summary()

# 解码z
z_decoded = decoder(z)

用于计算VAE损失的自定义层

from keras.metrics import binary_crossentropy

class CustomVariationalLayer(layers.Layer):
    
    def vae_loss(self, x, z_decoded):
        x = K.flatten(x)
        z_decoded = K.flatten(z_decoded)
        xent_loss = binary_crossentropy(x, z_decoded)
        k1_loss = -5e-4 * K.mean(1 + z_log_variance - K.square(z_mean) - K.exp(z_log_variance), axis=-1)
        
        return K.mean(xent_loss + k1_loss)
    
    def call(self, inputs, **kwargs):
        x = inputs[0]
        z_decoded = inputs[1]
        loss = self.vae_loss(x, z_decoded)
        self.add_loss(loss, inputs=inputs)
        return x
    
y = CustomVariationalLayer()([input_img, z_decoded])

训练VAE

from keras.datasets import mnist


vae = Model(input_img, y)
vae.compile(optimizer='rmsprop', loss=None)
vae.summary()

(x_train, _), (x_test, y_test) = mnist.load_data()

x_train = x_train.astype('float32') / 255.
x_train = x_train.reshape(x_train.shape + (1, ))
x_test = x_test.astype('float32') / 255.
x_test = x_test.reshape(x_test.shape + (1, ))

vae.fit(x=x_train, y=None, shuffle=True, epochs=10, batch_size=batch_size, validation_data=(x_test, None))