深度学习第一枪——>MNIST

前言

最近在学TensorFlow，第一个比较完整的程序就是对MNIST数据集进行识别。

一、MNIST数据集简介

MNIST是一个简单的计算机视觉数据集，它包含手写数字的图像集：

 

数据集：

每张图像是28 * 28像素：

我们的任务是使用CNN训练一个能够进行识别的模型。

二、模型构建

1.加载数据

我们先要下载需要的数据集保存到程序所在目录，下载地址：http://yann.lecun.com/exdb/mnist/

import tensorflow as tf
from tensorflow.examples.tutorials.mnist import input_data


#加载数据
mnist = input_data.read_data_sets('/MNIST_data',one_hot=True)
X_train = mnist.train.images
y_train = mnist.train.labels
print(y_train.shape)
print(X_train.shape)
'''
(55000, 10)
(55000, 784)
'''
X_valid = mnist.validation.images
y_valid = mnist.validation.labels
X_test = mnist.test.images
y_test = mnist.test.labels

2.定义所需的参数

#构建图
X =tf.placeholder(tf.float32,shape=[None,28*28],name='X')
y = tf.placeholder(tf.float32,shape=[None,10],name='y')
#定义W
def weight_variable(shape):    
    #初始化为正态分布，标准差为0.1
    W = tf.truncated_normal(shape,stddev=0.1)
    return tf.Variable(W,name='weights')

#定义biases
def biases_variable(shape):
    #初始化为常数0.1
    b = tf.constant(0.1,shape=shape)
    return tf.Variable(b,name='biases')

#定义卷积
def conv2d(X,W):
    #X=[size,height,weight,channels],stribes步长，形式为【1，x,y,1】,padding是否0填充
    return tf.nn.conv2d(X,W,strides=[1,1,1,1],padding='SAME')

def max_pool_2x2(X):
    #池化内核2x2,步长2
    return tf.nn.max_pool(X,ksize=[1,2,2,1],strides=[1,2,2,1],padding="SAME")

3.构建神经网络

我们使用简化版的LeNet-5模型，组成为输入-卷积-最大池化-卷积-最大池化-全连接-全连接输出，为抑制过拟合，在全连接层使用dropout

#第一层卷积
#前两个维度是卷积核大小，接着是输入的通道数目，最后是输出的通道数目。 
#而对于每一个输出通道都有一个对应的偏置量
W_conv1 = weight_variable([5,5,1,32])  
b_conv1 = biases_variable([32])
#将原始X转化成【height,width,channels】形状
X_image = tf.reshape(X,[-1,28,28,1])
conv1 = tf.nn.relu(conv2d(X_image,W_conv1) + b_conv1)

#第一层池化
pool1 = max_pool_2x2(conv1)

#第二层卷积
W_conv2 = weight_variable([5,5,32,64])
b_conv2 = biases_variable([64])
conv2 = tf.nn.relu(conv2d(pool1,W_conv2) + b_conv2)

#第二层池化
pool2 = max_pool_2x2(conv2)

#第三层全连接
#28两次池化7，一共64个通道，创建1024个神经元
W_fc1 = weight_variable([7*7*64,1024])
b_fc1 = biases_variable([1024])
pool2_flat = tf.reshape(pool2,shape=[-1,7*7*64])
h_fc1 = tf.nn.relu(tf.matmul(pool2_flat,W_fc1)+b_fc1)

#第三层加入dropout
keep_prob = tf.placeholder('float')
h_fc1_drop = tf.nn.dropout(h_fc1,keep_prob)    

#输出层
W_fc2 = weight_variable([1024,10])
b_fc2 = biases_variable([10])
y_predict = tf.add(tf.matmul(h_fc1_drop,W_fc2),b_fc2)

4.损失函数

使用tf.nn.softmax_entropy_with_logits计算交叉熵，注意它以one-hot形式获取标记，使用速度比较快的Adam算法进行梯度下降

xentory = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(labels=y,logits=y_predict))
training_op = tf.train.AdamOptimizer(learning_rate=0.01).minimize(xentory)

5.评价指标

#评价指标
correct_prediction = tf.equal(tf.argmax(y_predict,1), tf.argmax(y,1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float")) 

6.执行阶段

init = tf.global_variables_initializer()

#执行图
#运行400轮次
n_epochs = 400
#每次训练取小批次50个
batch_size = 50
with tf.Session() as sess:
    init.run()
    for epoch in range(400):
        for iteration in range(mnist.train.num_examples//batch_size):
            #mnist.train.next_batch函数根据batch_size大小划分出小批次
            X_batch,y_batch = mnist.train.next_batch(batch_size)
            #传入数据，在训练阶段保持dropout
            sess.run(training_op,feed_dict={X:X_batch,y:y_batch,keep_prob:0.5})
        #每隔40轮打印训练准确度
        if epoch % 40 == 0:
            accuracy_train = sess.run(accuracy,feed_dict={X:X_batch,y:y_batch,keep_prob:1.0})
            print('epoch:',epoch,'accuracy_train:',accuracy_train)
    #最后打印在测试集的准确度
    #在测试阶段关闭dropout
    accuracy_test = sess.run(accuracy,feed_dict={X:X_test,y:y_test,keep_prob:1.0})
    print('accuracy_tets:',accuracy_test)

7、结果

epoch: 0 accuracy_train: 0.94
epoch: 40 accuracy_train: 1.0
epoch: 80 accuracy_train: 1.0
epoch: 120 accuracy_train: 0.94
epoch: 160 accuracy_train: 1.0
epoch: 200 accuracy_train: 1.0
epoch: 240 accuracy_train: 0.96
epoch: 280 accuracy_train: 0.9
epoch: 320 accuracy_train: 0.98
epoch: 360 accuracy_train: 0.88
accuracy_tets: 0.9532

有4%的误差，不算太好。最好的误差在0.21%。

二 Keras

是不是感觉网络构建起来挺麻烦的，我们使用高级封装keras来简化代码：

import tensorflow as tf
from tensorflow import keras
import numpy as np


#加载数据
#[60000,28,28],[60000]  [10000,28,28],[10000]
(X_train,y_train),(X_test,y_test) = keras.datasets.mnist.load_data()

X_train = X_train.reshape(X_train.shape[0],28,28,1)
X_test = X_test.reshape(X_test.shape[0],28,28,1)
#归一化
X_train = X_train/255
X_test = X_test/255
#独热化
#[60000,10]
y_train = keras.utils.to_categorical(y_train,10)
#[10000,10]
y_test = keras.utils.to_categorical(y_test,10)

#构建层
input_shape = (28,28,1)
#使用顺序结构
model = keras.Sequential()
#卷积1 
model.add(keras.layers.Conv2D(32,kernel_size=(3,3),padding='SAME',
                              activation='relu',input_shape=input_shape))
#卷积2
model.add(keras.layers.Conv2D(64,kernel_size=(3,3),padding='SAME',
                              activation='relu'))
#池化1
model.add(keras.layers.MaxPool2D(strides=(2,2),padding='SAME'))
#dropout
model.add(keras.layers.Dropout(0.25))
model.add(keras.layers.Flatten())
#全连接1
model.add(keras.layers.Dense(128,activation='relu'))
#dropout
model.add(keras.layers.Dropout(0.5))
#输出层
model.add(keras.layers.Dense(10,activation='softmax'))

#编译 优化器Adadelta,loss:多分类，评价：精确度
model.compile(optimizer=keras.optimizers.Adadelta(),
              loss=keras.losses.categorical_crossentropy,
              metrics=['accuracy'])
#训练 批次大小为128，一共12轮
model.fit(X_train,y_train,batch_size=128,epochs=12)
#测试 
score = model.evaluate(X_test,y_test,batch_size=128)
print(score)

结果：

Epoch 1/12
60000/60000 [==============================] - 385s 6ms/sample - loss: 0.2549 - acc: 0.9217
Epoch 2/12
60000/60000 [==============================] - 377s 6ms/sample - loss: 0.0862 - acc: 0.9744

可以看到在第二轮时，精确度达到了97%