Tensorlayer学习笔记——多层神经网络

一、简单粗暴，先看代码

import tensorflow as tf
import tensorlayer as tl

sess = tf.InteractiveSession()
# 导入数据
X_train, y_train, X_val, y_val, X_test, y_test = tl.files.load_mnist_dataset(shape=(-1,784))

# 定义placeholder
x = tf.placeholder(tf.float32, shape=[None, 784], name='x')
y_ = tf.placeholder(tf.int64, shape = [None, ], name='y_')

# 定义模型
network = tl.layers.InputLayer(inputs=x, name='input_layer')
network = tl.layers.DropoutLayer(network, keep=0.5, name='dropout1')
network = tl.layers.DenseLayer(network, n_units=1024, act=tf.nn.relu, name='relu1')
network = tl.layers.DropoutLayer(network, keep=0.5, name='dropout2')
network = tl.layers.DenseLayer(network, n_units=512, act=tf.nn.relu, name='relu2')
network = tl.layers.DropoutLayer(network, keep=0.5, name='dropout3')
network = tl.layers.DenseLayer(network, n_units=10, act=tf.identity, name='output_layer')

# 定义损失函数
y = network.outputs
cost = tl.cost.cross_entropy(y, y_, name='cost')
correct_prediction = tf.equal(tf.arg_max(y, 1), y_)
acc = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
y_op = tf.arg_max(tf.nn.softmax(y), 1)

# 定义优化器
train_param = network.all_params
train_op = tf.train.AdamOptimizer(learning_rate=0.0001, use_locking=False).minimize(cost, var_list=train_param)

# tensorboard
acc_summ = tf.summary.scalar('acc', acc)
cost_summ = tf.summary.scalar('cost', cost)
summary = tf.summary.merge_all()
writer = tf.summary.FileWriter('./logs')
writer.add_graph(sess.graph)

#初始化参数
tl.layers.initialize_global_variables(sess)

# 列出模型信息
network.print_layers()
network.print_params()

# 训练模型
tl.utils.fit(sess, network, train_op, cost, X_train, y_train, x, y_,
             acc=acc, batch_size=512, n_epoch=100, print_freq=10,
             X_val=X_val, y_val=y_val,eval_train=False, tensorboard=True)

# 评估
tl.utils.test(sess, network, acc, X_test, y_test, x, y_, batch_size=None, cost=cost)

# 保存模型
tl.files.save_npz(network.all_params, name='model.npz')
sess.close()

二、细嚼慢咽，慢慢学习

Tensorlayer简单程度可以说是介于Keras和Tensorflow之间的，简化了Tensorflow很多繁琐的步骤。

1、第一步依然是建立会话

tf.InteractiveSession()

2、第二步导入数据，这里以MNIST为例

X_train, y_train, X_val, y_val, X_test, y_test = tl.files.load_mnist_dataset(shape=(-1,784))

第一次运行需要下载，再一次运行不需要下载

3、第三步，定义占位符，分别为网络的输入和目标输出

x = tf.placeholder(tf.float32, shape=[None, 784], name='x')
y_ = tf.placeholder(tf.int64, shape = [None, ], name='y_')

4、第四步，建立模型

network = tl.layers.InputLayer(inputs=x, name='input_layer')
network = tl.layers.DropoutLayer(network, keep=0.5, name='dropout1')
network = tl.layers.DenseLayer(network, n_units=1024, act=tf.nn.relu, name='relu1')
network = tl.layers.DropoutLayer(network, keep=0.5, name='dropout2')
network = tl.layers.DenseLayer(network, n_units=512, act=tf.nn.relu, name='relu2')
network = tl.layers.DropoutLayer(network, keep=0.5, name='dropout3')
network = tl.layers.DenseLayer(network, n_units=10, act=tf.identity, name='output_layer')

这里可以看出，风格和Keras很像，像搭积木一样堆积层就行。这里网络的结构是一个输入层，两个节点为1024、512的隐藏层，一个10节点的输出层，10个节点表示One-Hot编码。每层之间加入Dropout层，节点保留率为0.5。隐藏层的激活函数为relu函数。注意输出层的激活函数为tf.identity，DenseLayer的默认激活函数就是tf.identity，可以理解为线性激活函数，即输入输出相同。

5、第五步，定义损失函数

↓得到网络的输出

y = network.outputs

↓计算交叉熵损失，这个API调用tf的函数在内部实现了softmax

cost = tl.cost.cross_entropy(y, y_, name='cost')

↓判断预测结果和真实标签是否相同，得到的是一系列的布尔型Tensor

correct_prediction = tf.equal(tf.arg_max(y, 1), y_)

↓计算准确率

acc = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))

↓y是one-hot形式，y_op得到的是代表类别的索引

y_op = tf.arg_max(tf.nn.softmax(y), 1)

6、第六步，定义优化器

train_param = network.all_params
train_op = tf.train.AdamOptimizer(learning_rate=0.0001, use_locking=False).minimize(cost, var_list=train_param)

优化所有参数，最小化cost损失，采用Adam优化器

7、第七步，开启Tensorboard

acc_summ = tf.summary.scalar('acc', acc)
cost_summ = tf.summary.scalar('cost', cost)
summary = tf.summary.merge_all()
writer = tf.summary.FileWriter('./logs')
writer.add_graph(sess.graph)

这里监控准确率acc和损失cost

8、第八步，初始化所有参数

tl.layers.initialize_global_variables(sess)

9、第九步，列出模型的信息

network.print_layers()
network.print_params()

包括网络结构信息和参数信息

10、第十步，训练模型

tl.utils.fit(sess, network, train_op, cost, X_train, y_train, x, y_,
             acc=acc, batch_size=512, n_epoch=100, print_freq=10,
             X_val=X_val, y_val=y_val,eval_train=False, tensorboard=True)

输入会话sess、要训练的网络network、优化器train_op、损失cost、训练集、占位符、验证集，mini-batch大小为512，迭代100轮，每10轮打印一次训练信息。eval_train=False表示不验证训练集，如果要开启Tensorboard，则tensorboard=Ture。

11、第十一步，评估测试集

tl.utils.test(sess, network, acc, X_test, y_test, x, y_, batch_size=None, cost=cost)

12、第十二步，保存模型，关闭会话

tl.files.save_npz(network.all_params, name='model.npz')
sess.close()

三、飞起来吧

运行，输出如下：

Load or Download MNIST > data/mnist/
data/mnist/train-images-idx3-ubyte.gz
2017-12-11 21:30:58.096836: I C:\tf_jenkins\home\workspace\rel-win\M\windows\PY\35\tensorflow\core\platform\cpu_feature_guard.cc:137] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX
data/mnist/t10k-images-idx3-ubyte.gz
  [TL] InputLayer  input_layer: (?, 784)
  [TL] DropoutLayer dropout1: keep:0.500000 is_fix:False
  [TL] DenseLayer  relu1: 1024 relu
  [TL] DropoutLayer dropout2: keep:0.500000 is_fix:False
  [TL] DenseLayer  relu2: 512 relu
  [TL] DropoutLayer dropout3: keep:0.500000 is_fix:False
  [TL] DenseLayer  output_layer: 10 identity
WARNING:tensorflow:From E:/Machine_Learning/TensorLayer_code/mlp/mnist.py:24: arg_max (from tensorflow.python.ops.gen_math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `argmax` instead
  layer   0: dropout1/mul:0       (?, 784)           float32
  layer   1: relu1/Relu:0         (?, 1024)          float32
  layer   2: dropout2/mul:0       (?, 1024)          float32
  layer   3: relu2/Relu:0         (?, 512)           float32
  layer   4: dropout3/mul:0       (?, 512)           float32
  layer   5: output_layer/Identity:0 (?, 10)            float32
  param   0: relu1/W:0            (784, 1024)        float32_ref (mean: 2.0586548998835497e-05, median: -7.415843720082194e-05, std: 0.08802532404661179)   
  param   1: relu1/b:0            (1024,)            float32_ref (mean: 0.0               , median: 0.0               , std: 0.0               )   
  param   2: relu2/W:0            (1024, 512)        float32_ref (mean: -7.740945875411853e-05, median: -7.019848271738738e-05, std: 0.08793578296899796)   
  param   3: relu2/b:0            (512,)             float32_ref (mean: 0.0               , median: 0.0               , std: 0.0               )   
  param   4: output_layer/W:0     (512, 10)          float32_ref (mean: -0.0003667508135549724, median: 0.0003241676895413548, std: 0.08744712173938751)   
  param   5: output_layer/b:0     (10,)              float32_ref (mean: 0.0               , median: 0.0               , std: 0.0               )   
  num of params: 1333770
Setting up tensorboard ...
[!] logs/ exists ...
Param name  relu1/W:0
Param name  relu1/b:0
Param name  relu2/W:0
Param name  relu2/b:0
Param name  output_layer/W:0
Param name  output_layer/b:0
Finished! use $tensorboard --logdir=logs/ to start server
Start training the network ...
Epoch 1 of 100 took 28.844650s
   val loss: 0.782603
   val acc: 0.752262
Epoch 10 of 100 took 27.462571s
   val loss: 0.433058
   val acc: 0.898335

在项目目录下回生成一个logs文件夹，里面生成了tensorboard的events文件，在项目目录下运行cmd，输入tensorboard --logdir=logs/，根据提示打开浏览器输入网址，即可实时监控acc和cost如下图所示（如果打不开尝试一下谷歌浏览器）：