Tensorflow之CNN实践

最近刚学习完tensorflow cnn并用它跑了个实验。在tensorflow官网有关于手写数据集识别MNIST的实验，链接如下
https://www.tensorflow.org/versions/r0.11/tutorials/mnist/pros/index.html

与上一篇NN相比，CNN的难度体现在以下两点：
1. 对输入数据的处理
2. 网络中增加了更多的超参数

相比于NN，CNN多了卷积运算，因此需要定义卷积运算：

def conv_batch_normalization(x):
    mean, variance = tf.nn.moments(x, axes=[0, 1, 2])
    return tf.nn.batch_normalization(x, mean, variance, None, None, 0.0001)

注意输入样本的数据依然是像素点排列的格式，三通道彩色图片盒黑白图片仅是最后一个参数的不同：

x_reshaped = tf.reshape(x, [-1, input_data.IMAGE_WIDTH, input_data.IMAGE_HEIGHT, 3])

定义一个五层卷积和三层全连接的神经网络

# First convolutional layer, (224, 224, 3) to (56, 56, 48)
W_conv1 = weight_variable([11, 11, 3, 48])
b_conv1 = bias_variable([48])

h_conv1 = tf.nn.relu(conv2d(x_reshaped, W_conv1, [1, 4, 4, 1]) + b_conv1)

# Second convolutional layer, (56, 56, 48) to (28, 28, 128)
W_conv2 = weight_variable([5, 5, 48, 128])
b_conv2 = bias_variable([128])

h_conv2 = tf.nn.relu(conv2d(h_conv1, W_conv2, [1, 1, 1, 1]) + b_conv2)
h_pool2 = max_pool_2x2(h_conv2)

# Third convolutional layer, (28, 28, 128) to (14, 14, 192)
W_conv3 = weight_variable([3, 3, 128, 192])
b_conv3 = bias_variable([192])

h_conv3 = tf.nn.relu(conv2d(h_pool2, W_conv3, [1, 1, 1, 1]) + b_conv3)
h_pool3 = max_pool_2x2(h_conv3)

# Fourth convolutional layer, (14, 14, 192) to (14, 14, 192)
W_conv4 = weight_variable([3, 3, 192, 192])
b_conv4 = bias_variable([192])

h_conv4 = tf.nn.relu(conv2d(h_pool3, W_conv4, [1, 1, 1, 1]) + b_conv4)

# Fifth convolutional layer, (14, 14, 192) to (14, 14, 128)
W_conv5 = weight_variable([3, 3, 192, 128])
b_conv5 = bias_variable([128])

h_conv5 = tf.nn.relu(conv2d(h_conv4, W_conv5, [1, 1, 1, 1]) + b_conv5)

# First fully-connected layer
W_fc1 = relu_weight_variable([14 * 14 * 128, 512])
b_fc1 = bias_variable([512])

h_conv5_flat = tf.reshape(h_conv5, [-1, 14 * 14 * 128])
#h_fc1 = tf.nn.relu(fc_batch_normalization(tf.matmul(h_conv5_flat, W_fc1) + b_fc1))
h_fc1 = tf.nn.relu(tf.matmul(h_conv5_flat, W_fc1) + b_fc1)

keep_prob = tf.placeholder(tf.float32)
h_fc1_drop = tf.nn.dropout(h_fc1, keep_prob)

# Second fully-connected layer
W_fc2 = relu_weight_variable([512, 512])
b_fc2 = bias_variable([512])

#h_fc2 = tf.nn.relu(fc_batch_normalization(tf.matmul(h_fc1_drop, W_fc2) + b_fc2))
h_fc2 = tf.nn.relu(tf.matmul(h_fc1_drop, W_fc2) + b_fc2)
h_fc2_drop = tf.nn.dropout(h_fc2, keep_prob)

# Third fully-connected layer
W_fc3 = relu_weight_variable([512, num_classes])
b_fc3 = bias_variable([num_classes])

输出层：

y_score = tf.matmul(h_fc2_drop, W_fc3) + b_fc3
y_logit = tf.nn.softmax(y_score)

接下来再定义training cost, training gradient 就可以了。关于超参数的调试，没有一定准确的经验，通用的办法是扫描很多参数，从中挑选合适的一组。

代码参考：
https://github.com/TheoKanning/ImageNet-Classifier
修改数据文件夹即可使用。