Tensorlayer学习笔记——卷积神经网络

本文链接：https://blog.csdn.net/u014157632/article/details/78796493

一、先看代码

import tensorflow as tf
import tensorlayer as tl

sess = tf.InteractiveSession()
# 导入数据
X_train, y_train, X_val, y_val, X_test, y_test = tl.files.load_mnist_dataset(shape=(-1, 28, 28, 1))

# 定义placeholder
x = tf.placeholder(tf.float32, shape=[None, 28, 28, 1], name='x')
y_ = tf.placeholder(tf.int64, shape = [None, ], name='y_')

# 建立模型
network = tl.layers.InputLayer(inputs=x, name='input_layer')
network = tl.layers.Conv2dLayer(network, act=tf.nn.relu, shape=[5, 5, 1, 32], strides=[1, 1, 1, 1], padding='SAME', name='cnn_layer1')
network = tl.layers.PoolLayer(network, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='SAME', pool=tf.nn.max_pool, name='pool_layer1')

network = tl.layers.Conv2dLayer(network, act=tf.nn.relu, shape=[5, 5, 32, 64], strides=[1, 2, 2, 1], padding='SAME', name='cnn_layer2')
network = tl.layers.PoolLayer(network, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='SAME', pool=tf.nn.max_pool, name='pool_layer2')

network = tl.layers.FlattenLayer(network, name='flatten_layer')
network = tl.layers.DropoutLayer(network, keep=0.5, name='drop1')

network = tl.layers.DenseLayer(network, n_units=256, act=tf.nn.relu, name='fc1')
network = tl.layers.DropoutLayer(network, keep=0.5, name='drop2')

network = tl.layers.DenseLayer(network, n_units=10, act=tf.identity, name='output_layer')

# 定义损失函数
y = network.outputs
cost = tl.cost.cross_entropy(y, y_, name='cost')
correct_prediction = tf.equal(tf.arg_max(y, 1), y_)
acc = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
y_op = tf.arg_max(tf.nn.softmax(y), 1)

# 定义优化器
train_param = network.all_params
train_op = tf.train.AdamOptimizer(learning_rate=0.0001, use_locking=False).minimize(cost, var_list=train_param)

# tensorboard
acc_summ = tf.summary.scalar('acc', acc)
cost_summ = tf.summary.scalar('cost', cost)
summary = tf.summary.merge_all()
writer = tf.summary.FileWriter('./logs')
writer.add_graph(sess.graph)

# 初始化参数
tl.layers.initialize_global_variables(sess)

# 列出模型信息
network.print_layers()
network.print_params()

# 训练模型
tl.utils.fit(sess, network, train_op, cost, X_train, y_train, x, y_,
             acc=acc, batch_size=512, n_epoch=100, print_freq=10,
             X_val=X_val, y_val=y_val, eval_train=False, tensorboard=True)

# 评估
tl.utils.test(sess, network, acc, X_test, y_test, x, y_, batch_size=None, cost=cost)

# 保存模型
tl.files.save_npz(network.all_params, name='model.npz')
sess.close()

二、看看与多层神经网络不同的地方

上篇博文介绍了Tensorlayer的多层神经网络实现，这次实现卷积神经网络除了第二、三、四步不同外，其他都一样，在此不再介绍，下面直接解释第二三四步

1、第二步，导入MNIST数据

X_train, y_train, X_val, y_val, X_test, y_test = tl.files.load_mnist_dataset(shape=(-1, 28, 28, 1))

因为MNIST原始数据是一张图片数据是展开的，即784维的向量，卷积神经网络输入的是二维图像，将其reshape成28*28*1，MNIST是灰度图，通道数为1

2、第三步，定义占位符

x = tf.placeholder(tf.float32, shape=[None, 28, 28, 1], name='x')
y_ = tf.placeholder(tf.int64, shape = [None, ], name='y_')

同理x的shape是28*28*1

3、第四步，建立模型

↓首先定义输入层

network = tl.layers.InputLayer(inputs=x, name='input_layer')

↓定义一个卷积层：

network = tl.layers.Conv2dLayer(network, act=tf.nn.relu, shape=[5, 5, 1, 32], strides=[1, 1, 1, 1], padding='SAME', name='cnn_layer1')

激活函数为relu函数。shape参数为卷积核的形状，前两个代表过滤器大小，第三个表示前一层的通道数，也即每个过滤器的通道数，第四个表示过滤器的个数，此例则是5*5*1大小的过滤器，一共有32个这样的过滤器，卷积后输出有32个通道。strides参数为步长，四个元素分别代表样本数、长、宽、通道四个维度的步长。一般情况下图像长宽的步长就行，此处步长为1。padding参数有两个选项，SAME和VALID。

↓定义一个池化层

network = tl.layers.PoolLayer(network, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='SAME', pool=tf.nn.max_pool, name='pool_layer1')

ksize参数为池化窗口大小，和stride一样每个元素与输入各维度对应，此处为2*2大小的池化。stride、padding同上。此处选择最大池化

一个典型的卷积层定义结束，接下来按此方法定义余下各层：

network = tl.layers.Conv2dLayer(network, act=tf.nn.relu, shape=[5, 5, 32, 64], strides=[1, 2, 2, 1], padding='SAME', name='cnn_layer2')
network = tl.layers.PoolLayer(network, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='SAME', pool=tf.nn.max_pool, name='pool_layer2')

network = tl.layers.FlattenLayer(network, name='flatten_layer')
network = tl.layers.DropoutLayer(network, keep=0.5, name='drop1')

network = tl.layers.DenseLayer(network, n_units=256, act=tf.nn.relu, name='fc1')
network = tl.layers.DropoutLayer(network, keep=0.5, name='drop2')

最后依然是节点为10的输出层：

network = tl.layers.DenseLayer(network, n_units=10, act=tf.identity, name='output_layer')

其他部分请参考上篇博文

三、跑一跑

运行，输出如下：

Load or Download MNIST > data/mnist/
data/mnist/train-images-idx3-ubyte.gz
data/mnist/t10k-images-idx3-ubyte.gz
  [TL] InputLayer  input_layer: (?, 28, 28, 1)
  [TL] Conv2dLayer cnn_layer1: shape:[5, 5, 1, 32] strides:[1, 1, 1, 1] pad:SAME act:relu
  [TL] PoolLayer   pool_layer1: ksize:[1, 2, 2, 1] strides:[1, 2, 2, 1] padding:SAME pool:max_pool
  [TL] Conv2dLayer cnn_layer2: shape:[5, 5, 32, 64] strides:[1, 2, 2, 1] pad:SAME act:relu
  [TL] PoolLayer   pool_layer2: ksize:[1, 2, 2, 1] strides:[1, 2, 2, 1] padding:SAME pool:max_pool
  [TL] FlattenLayer flatten_layer: 1024
  [TL] DropoutLayer drop1: keep:0.500000 is_fix:False
  [TL] DenseLayer  fc1: 256 relu
  [TL] DropoutLayer drop2: keep:0.500000 is_fix:False
  [TL] DenseLayer  output_layer: 10 identity
WARNING:tensorflow:From E:/Machine_Learning/TensorLayer_code/cnn/cnn.py:31: arg_max (from tensorflow.python.ops.gen_math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use `argmax` instead
  layer   0: cnn_layer1/Relu:0    (?, 28, 28, 32)    float32
  layer   1: pool_layer1:0        (?, 14, 14, 32)    float32
  layer   2: cnn_layer2/Relu:0    (?, 7, 7, 64)      float32
  layer   3: pool_layer2:0        (?, 4, 4, 64)      float32
  layer   4: flatten_layer:0      (?, 1024)          float32
  layer   5: drop1/mul:0          (?, 1024)          float32
  layer   6: fc1/Relu:0           (?, 256)           float32
  layer   7: drop2/mul:0          (?, 256)           float32
  layer   8: output_layer/Identity:0 (?, 10)            float32
  param   0: cnn_layer1/W_conv2d:0 (5, 5, 1, 32)      float32_ref (mean: -2.510603553673718e-05, median: 0.0005096496315672994, std: 0.018007082864642143)   
  param   1: cnn_layer1/b_conv2d:0 (32,)              float32_ref (mean: 0.0               , median: 0.0               , std: 0.0               )   
  param   2: cnn_layer2/W_conv2d:0 (5, 5, 32, 64)     float32_ref (mean: -2.960372694360558e-05, median: -0.00010261016723234206, std: 0.017551297321915627)   
  param   3: cnn_layer2/b_conv2d:0 (64,)              float32_ref (mean: 0.0               , median: 0.0               , std: 0.0               )   
  param   4: fc1/W:0              (1024, 256)        float32_ref (mean: 1.893113676487701e-06, median: 0.00015916726260911673, std: 0.08775801211595535)   
  param   5: fc1/b:0              (256,)             float32_ref (mean: 0.0               , median: 0.0               , std: 0.0               )   
  param   6: output_layer/W:0     (256, 10)          float32_ref (mean: 0.0018607772653922439, median: 0.0004102111852262169, std: 0.08700685948133469)   
  param   7: output_layer/b:0     (10,)              float32_ref (mean: 0.0               , median: 0.0               , std: 0.0               )   
  num of params: 317066
Setting up tensorboard ...
[!] logs/ exists ...
Param name  cnn_layer1/W_conv2d:0
Param name  cnn_layer1/b_conv2d:0
Param name  cnn_layer2/W_conv2d:0
Param name  cnn_layer2/b_conv2d:0
Param name  fc1/W:0
Param name  fc1/b:0
Param name  output_layer/W:0
Param name  output_layer/b:0
Finished! use $tensorboard --logdir=logs/ to start server
Start training the network ...
Epoch 1 of 100 took 140.208019s
   val loss: 1.406113
   val acc: 0.737664
Epoch 10 of 100 took 119.369828s
   val loss: 0.109499
   val acc: 0.969984
Epoch 20 of 100 took 123.663073s
   val loss: 0.073103
   val acc: 0.978721