文章目录
MINIST 手写数字识别数据集，由Yann LeCun所收集，CNN创始人
train：55000
trainval: 5000
test：10000
图片大小28*28，灰度图
本文用 Convolutional Neural Network（CNN）进行MNIST识别，CNN由Yann LeCun 提出，与MLP的区别在于，前面用卷积进行了特征提取，后面全连接层和Multilayer Perceptron（MLP）一样，MLP的demo可以查看这篇博客【TensorFlow-MLP】MNIST，本文的套路都是基于这篇博客的。精简版可以参考 《Tensorflow 搭建自己的神经网络 (莫烦 Python 教程)》learning notes。
1 database
import tensorflow as tf
import tensorflow.examples.tutorials.mnist.input_data as input_data
mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)
2 建立共享函数
2.1 weight
tf.truncated_normal 以随机化的方式初始化权重，然后用 tf.Variable 建立TensorFlow变量
def weight(shape):
return tf.Variable(tf.truncated_normal(shape, stddev=0.1),
name ='W')
2.2 bias
def bias(shape):
return tf.Variable(tf.constant(0.1, shape=shape)
, name = 'b')
2.3 conv2d
x必须是4维张量
strides = [1,stride,stride,1]
padding 为 same 会自动padding 0
def conv2d(x, W):
return tf.nn.conv2d(x, W, strides=[1,1,1,1],
padding='SAME')
2.4 max_pool_2x2
ksize = [1,height,width,1]
def max_pool_2x2(x):
return tf.nn.max_pool(x, ksize=[1,2,2,1],
strides=[1,2,2,1],
padding='SAME')
2 Build Model
2.1 建立输入层
名字为Input_Layer
with tf.name_scope('Input_Layer'):
x = tf.placeholder("float",shape=[None, 784]
,name="x")
x_image = tf.reshape(x, [-1, 28, 28, 1])
tf.name_scope（） 设置计算图的名字
shape=[None, 784]中 None 表示后续我们会传入很多图片，项数不固定，784是28*28 我们图片的大小
tf.reshape 第一维，-1是因为输入的图片个数不固定
二三维是图片的大小
第四维1因为是灰度图，如果是彩色要设置为3
2.2 建立卷积层1
名字为 C1_Conv
with tf.name_scope('C1_Conv'):
W1 = weight([5,5,1,16])# filter大小5×5，输入channel为1，输出为16
b1 = bias([16])
Conv1=conv2d(x_image, W1)+ b1
C1_Conv = tf.nn.relu(Conv1 )
2.3 建立池化层1
目的：
- 减少所需处理的数据量
- 让图像位置差异边变小
- 参数的数量和计算量下降（缓解过拟合）
名字为 C1_Pool，通过第一个pooling层后，28×28的图片变为14×14
with tf.name_scope('C1_Pool'):
C1_Pool = max_pool_2x2(C1_Conv)
2.4 建立卷积层2
名字为C2_Conv
with tf.name_scope('C2_Conv'):
W2 = weight([5,5,16,36]) #filter size 5×5，输入channel 16，输出36
b2 = bias([36])
Conv2=conv2d(C1_Pool, W2)+ b2
C2_Conv = tf.nn.relu(Conv2)
2.5 建立池化层2
名字 C2_Pool，14×14的图片变为7×7
with tf.name_scope('C2_Pool'):
C2_Pool = max_pool_2x2(C2_Conv)
2.6 建立全连接层
名字D_Flat
with tf.name_scope('D_Flat'):
D_Flat = tf.reshape(C2_Pool, [-1, 1764])
第一维是-1，因为后续会传入不限定项数的训练数据——数据图像，-1的功能是保持第一维不变
第二维 1764 = 7*7*36
2.7 建立隐藏层
名字 D_Hidden_Layer
with tf.name_scope('D_Hidden_Layer'):
W3= weight([1764, 128])# 输入1764，输出128
b3= bias([128])
D_Hidden = tf.nn.relu(
tf.matmul(D_Flat, W3)+b3)
D_Hidden_Dropout= tf.nn.dropout(D_Hidden,
keep_prob=0.8)
D_Hidden 进行drop out 操作，keep_prob=0.8 表示保留80% 的神经元
2.8 建立输出层
名字为 Output_Layer
with tf.name_scope('Output_Layer'):
W4 = weight([128,10])#输入128，输出10
b4 = bias([10])
y_predict= tf.nn.softmax(
tf.matmul(D_Hidden_Dropout,
W4)+b4)
2.9 定义训练方式
Keras 和 TensorFlow 定义训练方式不同
Keras：只需要使用model.compile 设置损失函数、优化器、并用 metrics 设置评估模型的方式
TensorFlow： 必须自行定义损失函数的公式、优化器和设置参数，并定义评估模型准确率的公式
with tf.name_scope("optimizer"):
y_label = tf.placeholder("float", shape=[None, 10],
name="y_label")
loss_function = tf.reduce_mean(
tf.nn.softmax_cross_entropy_with_logits
(logits=y_predict ,
labels=y_label))
optimizer = tf.train.AdamOptimizer(learning_rate=0.0001) \
.minimize(loss_function)
2.10 定义模型的准确率的方式
名字为 evaluate_model，one-hot编码，argmax输出最大值的位置，
with tf.name_scope("evaluate_model"):
# 计算每一项数据是否正確预测
# tf.equal,相等返回1，不等返回0
correct_prediction = tf.equal(tf.argmax(y_predict, 1),
tf.argmax(y_label, 1))
# 将計算预测正确结果，加权平均
# 先使用 tf.cast 转化为 float，再使用 tf.reduce_mean 将所有值平均
accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float"))
3 Training process
3.1 参数设置
trainEpochs = 30
batchSize = 100
totalBatchs = int(mnist.train.num_examples/batchSize)
print(totalBatchs)
epoch_list=[];accuracy_list=[];loss_list=[];
from time import time
startTime=time()
sess = tf.Session()
sess.run(tf.global_variables_initializer())
output
550
3.2 Train
# for each epoch
for epoch in range(trainEpochs):
# for each mini-batch
for i in range(totalBatchs):
# 调用读mini-batch的函数
batch_x, batch_y = mnist.train.next_batch(batchSize)
# 训练优化器，更新w，最小化损失函数，输入x和y
sess.run(optimizer,feed_dict={x: batch_x,y_label: batch_y})
# 在验证集上计算精度和误差
loss,acc = sess.run([loss_function,accuracy],
feed_dict={x: mnist.validation.images,
y_label: mnist.validation.labels})
# 显示训练结果并存入列表
epoch_list.append(epoch)
loss_list.append(loss)
accuracy_list.append(acc)
print("Train Epoch:", '%02d' % (epoch+1), "Loss=", \
"{:.9f}".format(loss)," Accuracy=",acc)
# 计算训练的时间
duration =time()-startTime
print("Train Finished takes:",duration)
output
Train Epoch: 01 Loss= 1.653820515 Accuracy= 0.825
Train Epoch: 02 Loss= 1.542834163 Accuracy= 0.9316
Train Epoch: 03 Loss= 1.522341847 Accuracy= 0.9488
Train Epoch: 04 Loss= 1.508871317 Accuracy= 0.9578
Train Epoch: 05 Loss= 1.503607273 Accuracy= 0.9624
Train Epoch: 06 Loss= 1.497657776 Accuracy= 0.9692
Train Epoch: 07 Loss= 1.495323420 Accuracy= 0.9692
Train Epoch: 08 Loss= 1.490671635 Accuracy= 0.973
Train Epoch: 09 Loss= 1.491126657 Accuracy= 0.9722
Train Epoch: 10 Loss= 1.487095952 Accuracy= 0.9764
Train Epoch: 11 Loss= 1.485939264 Accuracy= 0.9776
Train Epoch: 12 Loss= 1.484445453 Accuracy= 0.9782
Train Epoch: 13 Loss= 1.482881784 Accuracy= 0.98
Train Epoch: 14 Loss= 1.481629491 Accuracy= 0.9812
Train Epoch: 15 Loss= 1.480455637 Accuracy= 0.9824
Train Epoch: 16 Loss= 1.480812192 Accuracy= 0.9814
Train Epoch: 17 Loss= 1.478735685 Accuracy= 0.984
Train Epoch: 18 Loss= 1.477754354 Accuracy= 0.9856
Train Epoch: 19 Loss= 1.480426788 Accuracy= 0.9822
Train Epoch: 20 Loss= 1.478232026 Accuracy= 0.9842
Train Epoch: 21 Loss= 1.478392363 Accuracy= 0.9848
Train Epoch: 22 Loss= 1.475563645 Accuracy= 0.987
Train Epoch: 23 Loss= 1.477076173 Accuracy= 0.9846
Train Epoch: 24 Loss= 1.477659345 Accuracy= 0.9854
Train Epoch: 25 Loss= 1.477388263 Accuracy= 0.984
Train Epoch: 26 Loss= 1.477272630 Accuracy= 0.9852
Train Epoch: 27 Loss= 1.474917412 Accuracy= 0.9876
Train Epoch: 28 Loss= 1.474585891 Accuracy= 0.9878
Train Epoch: 29 Loss= 1.475056171 Accuracy= 0.9872
Train Epoch: 30 Loss= 1.474527240 Accuracy= 0.987
Train Finished takes: 77.65863227844238
3.3 可视化精度和损失变化
先看看损失函数
%matplotlib inline
import matplotlib.pyplot as plt
fig = plt.gcf()
fig.set_size_inches(5,3)
plt.plot(epoch_list, loss_list, label = 'loss')
plt.ylabel('loss')
plt.xlabel('epoch')
plt.legend(['loss'])
output
再看看精度变化
plt.plot(epoch_list, accuracy_list,label="accuracy" )
fig = plt.gcf()
fig.set_size_inches(5,3)
plt.ylim(0.8,1)
plt.ylabel('accuracy')
plt.xlabel('epoch')
plt.legend()
plt.show()
output
4 评估模型的准确性
查看测试集的大小
len(mnist.test.images)
output
10000
用训练的模型测试下性能
print("Accuracy:",
sess.run(accuracy,feed_dict={x: mnist.test.images,
y_label: mnist.test.labels}))
output
Accuracy: 0.9855
5 预测概率和结果
5.1 预测的概率
y_predict=sess.run(y_predict,
feed_dict={x: mnist.test.images})
看看前5项
y_predict[:5]
output
array([[3.02073045e-16, 2.95635516e-16, 2.74746163e-11, 1.37873916e-14,
1.98319444e-20, 3.49799479e-22, 4.14647587e-33, 1.00000000e+00,
7.10312555e-19, 7.26562098e-16],
[1.31700683e-12, 1.75494536e-10, 1.00000000e+00, 4.09089775e-14,
3.36607737e-24, 8.11432040e-25, 2.81481463e-16, 1.04421221e-22,
1.00114605e-13, 5.16368183e-25],
[7.71991552e-07, 9.99960899e-01, 8.24850375e-08, 6.61203448e-09,
2.96174749e-05, 9.90071393e-12, 2.44029543e-08, 2.58124288e-07,
8.30804674e-06, 7.27800459e-11],
[9.99999642e-01, 1.25004302e-16, 2.09856771e-10, 1.53727348e-17,
7.64048918e-16, 2.45482430e-16, 4.06873284e-07, 2.00636972e-11,
5.20325861e-13, 3.99716745e-14],
[2.58168181e-10, 1.29268818e-10, 1.05348274e-14, 5.35697674e-19,
9.99999881e-01, 8.84295946e-13, 5.98036228e-14, 3.60072305e-09,
4.24842461e-15, 1.09104448e-07]], dtype=float32)
5.2 预测的结果
prediction_result=sess.run(tf.argmax(y_predict,1),
feed_dict={x: mnist.test.images})
查看前10项
prediction_result[:10]
output
array([7, 2, 1, 0, 4, 1, 4, 9, 5, 9])
6 可视化预测结果
import numpy as np
def show_images_labels_predict(images,labels,prediction_result):
fig = plt.gcf()
fig.set_size_inches(8, 10)
for i in range(0, 10):
ax=plt.subplot(5,5, 1+i)
ax.imshow(np.reshape(images[i],(28, 28)),
cmap='binary')
ax.set_title("label=" +str(np.argmax(labels[i]))+
",predict="+str(prediction_result[i])
,fontsize=9)
plt.show()
调用
plot_images_labels_prediction(mnist.test.images,
mnist.test.labels,
prediction_result,0)
output
7 找出预测出错的结果
for i in range(500):
if prediction_result[i]!=np.argmax(mnist.test.labels[i]):
print("i="+str(i)+
" label=",np.argmax(mnist.test.labels[i]),
"predict=",prediction_result[i])
output
i=78 label= 9 predict= 8
i=247 label= 4 predict= 2
i=320 label= 9 predict= 1
i=340 label= 5 predict= 3
i=359 label= 9 predict= 4
i=445 label= 6 predict= 0
i=449 label= 3 predict= 5
i=492 label= 2 predict= 3
i=495 label= 8 predict= 0
可视化一下
def show_images_labels_predict_error(images,labels,prediction_result):
fig = plt.gcf()
fig.set_size_inches(8, 10)
i=0;j=0
while i<10:
if prediction_result[j]!=np.argmax(labels[j]):
ax=plt.subplot(5,5, 1+i)
ax.imshow(np.reshape(images[j],(28, 28)),
cmap='binary')
ax.set_title("j="+str(j)+
",l=" +str(np.argmax(labels[j]))+
",p="+str(prediction_result[j])
,fontsize=9)
i=i+1
j=j+1
plt.show()
调用
show_images_labels_predict_error(mnist.test.images,mnist.test.labels,prediction_result)
output
8 TensorBoard
用下面的代码，把要显示的计算图写入log文件
merged = tf.summary.merge_all()
train_writer = tf.summary.FileWriter('log/CNN',sess.graph)
如何打开TensorBoard可以参考【TensorFlow】程序设计模式 第5节（在本地访问服务器的log）
output
声明
声明：代码源于《TensorFlow+Keras深度学习人工智能实践应用》 林大贵版，引用、转载请注明出处，谢谢，如果对书本感兴趣，买一本看看吧！！！
1万+
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
