tensorlayer学习日志4_chapter3_3.3

最新推荐文章于 2024-05-11 09:35:34 发布

zouqiuyi

最新推荐文章于 2024-05-11 09:35:34 发布

阅读量344

点赞数

分类专栏： tensorflow tensorlayer 文章标签： tensorflow tensorlayer

本文链接：https://blog.csdn.net/weixin_42025210/article/details/81561451

版权

本文介绍了TensorLayer中自编码器的使用，重点探讨了稀疏自编码器与普通自编码器的区别。通过实验发现，ReLU激活函数的普通自编码器在重构效果上优于Sigmoid激活的稀疏自编码器，且ReLU的损失函数下降更快，有助于模型防止过拟合。

摘要由CSDN通过智能技术生成

第三章一开始就傻眼了~~！什么是正则化LP啊~然后果断回头看第二章看了几遍还是难理解，然后参考了这个博客，终于有点感觉了，https://vimsky.com/article/3852.html，其实所谓训练出来的模型，就是W和B的多种可能的集合，B是加减级别的，而W是乘除级别的，所以W的调参是更重要的。这整个第三章就是希望能在训练模型时，调出最好的参来，减少过多无效的和太有效W，从而增加模型的准确同时又要防止过拟合~~~而这调整的工具就是自编码器！

这里3.3节先要完成两个自编码器，一个是稀疏自编码器，由sigmoid函数进行，一个是普通自编码器，由relu函数进行

import tensorflow as tf
import tensorlayer as tl
import numpy as np


learning_rate = 0.0001
lambda_l2_w = 0.01
n_epochs = 200
batch_size =128
print_interval = 200
hidden_size = 196
input_size = 784
image_width = 28
model = 'sigmoid'
# model = 'relu'

x = tf.placeholder(tf.float32, shape=[None, 784], name='x')

print('~~~~~~~~~~~~~~~~build network~~~~~~~~~~~~~~~~~')

if model == 'relu':
	network = tl.layers.InputLayer(x, name='input')
	network = tl.layers.DenseLayer(network, hidden_size, tf.nn.relu, name='relu1')

	encoded_img = network.outputs
	recon_layer1 = tl.layers.DenseLayer(network, input_size, tf.nn.softplus, name='recon_layer1')

elif model == 'sigmoid':
	network = tl.layers.InputLayer(x, name='input')
	network = tl.layers.DenseLayer(network, hidden_size, tf.nn.sigmoid, name='sigmoid1')

	encoded_img = network.outputs
	recon_layer1 = tl.layers.DenseLayer(network, input_size, tf.nn.sigmoid, name='recon_layer1')

y = recon_layer1.outputs
train_params = recon_layer1.all_params[-4:]

mse = tf.reduce_sum(tf.squared_difference(y, x), 1)
mse = tf.reduce_mean(mse)

L2_w = tf.contrib.layers.l2_regularizer(lambda_l2_w)(train_params[0] + tf.contrib.layers.l2_regularizer(lambda_l2_w)(train_params[2]))

activation_out = recon_layer1.all_layers[-2]
L1_a = 0.001 * tf.reduce_mean(activation_out)

beta = 5
rho = 0.15
p_hat = tf.reduce_mean(activation_out, 0)
KLD = beta * tf.reduce_sum(rho * tf.log(tf.divide(rho, p_hat)) + (1-rho) * tf.log((1-rho)/(tf.subtract(float(1), p_hat))))

if model == 'sigmoid':
	cost = mse + L2_w + KLD
if model ==