Adrianna的专栏

Stay hungry, stay foolish.

【tensorflow学习】卷积神经网络CNN(mnist数据集)

一、 导入数据

from tensorflow.examples.tutorials.mnist import input_data
mnist = input_data.read_data_sets('MNIST_data/', one_hot=True)

每个example的类label以one-hot形式设定。

二、 运行TensorFlow的InteractiveSession

sess = tf.InteractiveSession()

Tensorflow依赖于一个高效的C++后端来进行计算。与后端的这个连接叫做session。一般而言,使用TensorFlow程序的流程是先创建一个图,然后在session中启动它。
InteractiveSession和普通的tf.Session()的区别在于,InteractiveSession能把自己作为默认的session,tf.Tensor.eval 和tf.Operation.run 就是用这个默认session运行ops的。
如果是普通的session,达到同样的效果则需要用‘with’。

二、 定义超参数

# hyperparams
learning_rate = 1e-4
batch_size = 50
step = 20000
display_step = 100
drop_out = 0.5

三、创建图(build graph)

3.1 声明图的输入(graph input)

# graph input
x = tf.placeholder(tf.float32, [None, 784]) #  none :不确定个batch
y = tf.placeholder(tf.float32, [None, 10])
keep_prob = tf.placeholder(tf.float32)

3.2 创建cnn模型(feedforward /prediction)

首先为了简便起见定义一些函数:

# Create some wrappers for simplicity
def weight_var(shape):
    return tf.Variable(tf.truncated_normal(shape, stddev=0.1))


def bias_var(shape):
    return tf.Variable(tf.random_normal(shape))


def conv2d(x, W, b):
    # conv with bias and 0-padding so images size stay the same
    # strides:[batch, height, width, channels], batch和channels一定为1
    x = tf.nn.conv2d(x, W, strides=[1, 1, 1, 1], padding='SAME')
    return tf.nn.bias_add(x, b)

def max_pool(x):
    # [batch, height, width, channels]
    return tf.nn.max_pool(x, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='SAME')

然后用以上的函数定义cnn模型结构(cov1+pool1+cov2+pool2+fc1+dropout+softmax):

# create model
def cnn(x, y, keep_prob):
    # Reshape input picture
    x = tf.reshape(x, [-1, 28, 28, 1])

    # 5x5 conv, 1 input, 32 outputs
    w_conv1 = weight_var([5, 5, 1, 32])
    b_conv1 = bias_var([32])

    # Convolution Layer
    h_conv1 = tf.nn.relu(conv2d(x, w_conv1, b_conv1))
    # Max Pooling (down-sampling)
    h_pool1 = max_pool(h_conv1) 

    # 5x5 conv, 32 inputs, 64 outputs
    w_conv2 = weight_var([5, 5, 32, 64])
    b_conv2 = bias_var([64]) 

    # Convolution Layer
    h_conv2 = tf.nn.relu(conv2d(h_pool1, w_conv2, b_conv2))
    # Max Pooling (down-sampling)
    h_pool2 = max_pool(h_conv2)

    # fully connected, 7*7*64 inputs, 1024 outputs
    w_fc1 = weight_var([7 * 7 * 64, 1024])
    b_fc1 = bias_var([1024])

    # Fully connected layer
    # Reshape conv2 output to fit fully connected layer input
    h_pool2_flat = tf.reshape(h_pool2, [-1, 7 * 7 * 64])
    h_fc1 = tf.nn.relu(tf.matmul(h_pool2_flat, w_fc1) + b_fc1)

    # Apply Dropout
    h_fc1_drop = tf.nn.dropout(h_fc1, keep_prob) 

    # 1024 inputs, 10 outputs (class prediction)
    w_fc2 = weight_var([1024, 10])
    b_fc2 = bias_var([10])

    # Output, class prediction
    logit = tf.matmul(h_fc1_drop, w_fc2) + b_fc2
    return logit

# construct model
pred = cnn(x, y, keep_prob)

3.3 定义loss function和优化方法(Optimization)

loss function 用交叉熵,优化方法用adam 而不用gradient descent,因为在mnist数据集上adam效果更好。

# define loss and optimizer
cross_entropy = tf.reduce_mean(
    tf.nn.softmax_cross_entropy_with_logits(labels=y, logits=pred))
optimizer = tf.train.AdamOptimizer(learning_rate).minimize(cross_entropy)

3.4 模型评估 (accuracy )

# evaluate model
correct_pred = tf.equal(tf.argmax(pred, 1), tf.argmax(y, 1))
accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.float32))

四、 用session.run 训练并测试模型

4.1 初始化所有变量

# init variables
sess.run(tf.global_variables_initializer())

4.2 训练模型

for i in range(step):
    batch = mnist.train.next_batch(batch_size)
    if i % display_step == 0:
        loss, acc = sess.run([cross_entropy, accuracy], feed_dict={
                             x: batch[0], y: batch[1], keep_prob: 1.0})
        print('step: %d' %
              i, " minibatch loss: {0:.6f},training accuracy:{1:.5f} ".format(loss, acc))
    optimizer.run(feed_dict={x: batch[0], y: batch[1], keep_prob: drop_out})#用默认session运行optimizer

4.3 测试模型

# test accuracy
print('Testing accuracy:', sess.run(accuracy, feed_dict={
      x: mnist.test.images[:500], y: mnist.test.labels[:500], keep_prob: 1.}))

五、测试结果

......
step: 17800  minibatch loss: 0.039849,training accracy:0.98000 
step: 17900  minibatch loss: 0.051081,training accracy:0.96000 
step: 18000  minibatch loss: 0.019745,training accracy:1.00000 
step: 18100  minibatch loss: 0.002456,training accracy:1.00000 
step: 18200  minibatch loss: 0.005077,training accracy:1.00000 
step: 18300  minibatch loss: 0.000313,training accracy:1.00000 
step: 18400  minibatch loss: 0.007955,training accracy:1.00000 
step: 18500  minibatch loss: 0.007918,training accracy:1.00000 
step: 18600  minibatch loss: 0.086002,training accracy:0.98000 
step: 18700  minibatch loss: 0.002512,training accracy:1.00000 
step: 18800  minibatch loss: 0.005236,training accracy:1.00000 
step: 18900  minibatch loss: 0.002420,training accracy:1.00000 
step: 19000  minibatch loss: 0.014115,training accracy:1.00000 
step: 19100  minibatch loss: 0.010140,training accracy:1.00000 
step: 19200  minibatch loss: 0.021536,training accracy:0.98000 
step: 19300  minibatch loss: 0.001117,training accracy:1.00000 
step: 19400  minibatch loss: 0.016043,training accracy:1.00000 
step: 19500  minibatch loss: 0.002531,training accracy:1.00000 
step: 19600  minibatch loss: 0.005710,training accracy:1.00000 
step: 19700  minibatch loss: 0.002422,training accracy:1.00000 
step: 19800  minibatch loss: 0.007958,training accracy:1.00000 
step: 19900  minibatch loss: 0.067153,training accracy:0.98000 
Testing accuracy: 0.994
[Finished in 306.8s]

六、 完整代码

# -*- coding: utf-8 -*-
# @Author: adrianna
# @Date:   2017-05-19 20:35:55
# @Last Modified by:   adrianna
# @Last Modified time: 2017-06-02 11:54:56

import tensorflow as tf
import os
from tensorflow.examples.tutorials.mnist import input_data

os.environ['TF_CPP_MIN_LOG_LEVEL'] = '2'

sess = tf.InteractiveSession()

# hyperparams
learning_rate = 1e-4
batch_size = 50
step = 20000
display_step = 100
drop_out = 0.5

# load data
mnist = input_data.read_data_sets('MNIST_data/', one_hot=True)

# graph input
x = tf.placeholder(tf.float32, [None, 784])
y = tf.placeholder(tf.float32, [None, 10])
keep_prob = tf.placeholder(tf.float32)


# Create some wrappers for simplicity
def weight_var(shape):
    return tf.Variable(tf.truncated_normal(shape, stddev=0.1))


def bias_var(shape):
    return tf.Variable(tf.random_normal(shape))


def conv2d(x, W, b):
    # conv with bias
    x = tf.nn.conv2d(x, W, strides=[1, 1, 1, 1], padding='SAME')
    return tf.nn.bias_add(x, b)


def max_pool(x):
    return tf.nn.max_pool(x, ksize=[1, 2, 2, 1], strides=[1, 2, 2, 1], padding='SAME')


# create model
def cnn(x, y, keep_prob):
    x = tf.reshape(x, [-1, 28, 28, 1])

    w_conv1 = weight_var([5, 5, 1, 32])
    b_conv1 = bias_var([32])

    h_conv1 = tf.nn.relu(conv2d(x, w_conv1, b_conv1))
    h_pool1 = max_pool(h_conv1)

    w_conv2 = weight_var([5, 5, 32, 64])
    b_conv2 = bias_var([64])

    h_conv2 = tf.nn.relu(conv2d(h_pool1, w_conv2, b_conv2))
    h_pool2 = max_pool(h_conv2)

    w_fc1 = weight_var([7 * 7 * 64, 1024])
    b_fc1 = bias_var([1024])

    h_pool2_flat = tf.reshape(h_pool2, [-1, 7 * 7 * 64])
    h_fc1 = tf.nn.relu(tf.matmul(h_pool2_flat, w_fc1) + b_fc1)

    h_fc1_drop = tf.nn.dropout(h_fc1, keep_prob)

    w_fc2 = weight_var([1024, 10])
    b_fc2 = bias_var([10])

    logit = tf.matmul(h_fc1_drop, w_fc2) + b_fc2

    return logit


# construct model
pred = cnn(x, y, keep_prob)
# define loss and optimizer
cross_entropy = tf.reduce_mean(
    tf.nn.softmax_cross_entropy_with_logits(labels=y, logits=pred))
optimizer = tf.train.AdamOptimizer(learning_rate).minimize(cross_entropy)

# evaluate model
correct_pred = tf.equal(tf.argmax(pred, 1), tf.argmax(y, 1))
accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.float32))

# init variables
sess.run(tf.global_variables_initializer())

for i in range(step):
    batch = mnist.train.next_batch(batch_size)
    if i % display_step == 0:
        # train_accuracy = accuracy.eval(feed_dict={
        #     x: batch[0], y: batch[1], keep_prob: 1.0})
        # print("step %d, training accuracy %g" % (i, train_accuracy))
        loss, acc = sess.run([cross_entropy, accuracy], feed_dict={
                             x: batch[0], y: batch[1], keep_prob: 1.0})
        print('step: %d' %
              i, " minibatch loss: {0:.6f},training accuracy:{1:.5f} ".format(loss, acc))
    optimizer.run(feed_dict={x: batch[0], y: batch[1], keep_prob: drop_out})


# test accuracy
print('Testing accuracy:', sess.run(accuracy, feed_dict={
      x: mnist.test.images[:500], y: mnist.test.labels[:500], keep_prob: 1.}))

参考:

  1. tensorflow 官网:https://www.tensorflow.org/get_started/mnist/pros
  2. https://github.com/aymericdamien/TensorFlow-Examples
阅读更多
版权声明:本文为博主原创文章,未经博主允许不得转载。 https://blog.csdn.net/u012223913/article/details/72844360
个人分类: tensorflow学习
所属专栏: tensorflow学习
想对作者说点什么? 我来说一句

没有更多推荐了,返回首页

加入CSDN,享受更精准的内容推荐,与500万程序员共同成长!
关闭
关闭