文章目录
MINIST 手写数字识别数据集，由Yann LeCun所收集，CNN创始人
- train：55000
- trainval: 5000
- test：10000
图片大小28*28，灰度图
本文用Multilayer Perceptron（MLP）的方法，来进行MINIST的识别，简洁版可以参考《Tensorflow 搭建自己的神经网络 (莫烦 Python 教程)》learning notes 中的5.1小节
1 database
1.1 下载数据集
import tensorflow as tf
import tensorflow.examples.tutorials.mnist.input_data as input_data
mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)
mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)
查看数据的大小
print('train',mnist.train.num_examples,
',validation',mnist.validation.num_examples,
',test',mnist.test.num_examples)
output
train 55000 ,validation 5000 ,test 10000
1.2 查看training data
print('train images:', mnist.train.images.shape,
'labels:', mnist.train.labels.shape)
len(mnist.train.images[0])
output
train images: (55000, 784) labels: (55000, 10)
784
1.3 显示图形
import matplotlib.pyplot as plt
def plot_image(image):
plt.imshow(image.reshape(28,28),cmap='binary')
plt.show()
调用
plot_image(mnist.train.images[0])
output
1.4 查看label
之前读数据的时候，one_hot=True，所以采用one-hot编码
mnist.train.labels[0]
output
array([ 0., 0., 0., 0., 0., 0., 0., 1., 0., 0.])
查看哪一项为1
import numpy as np
np.argmax(mnist.train.labels[0])
output
7
说明此图片为数字7
1.5 查看validation
print('validation images:', mnist.validation.images.shape,
'labels:', mnist.validation.labels.shape)
output
validation images: (5000, 784) labels: (5000, 10)
可视化显示一下
函数 plot_images_labels_prediction
import matplotlib.pyplot as plt
def plot_images_labels_prediction(images,labels,
prediction,idx,num=10):
fig = plt.gcf()
fig.set_size_inches(12, 14)
if num>25: num=25
for i in range(0, num):
ax=plt.subplot(5,5, 1+i)
ax.imshow(np.reshape(images[idx],(28, 28)),
cmap='binary')
title= "label=" +str(np.argmax(labels[idx]))
if len(prediction)>0:
title+=",predict="+str(prediction[idx])
ax.set_title(title,fontsize=10)
ax.set_xticks([]);ax.set_yticks([])
idx+=1
plt.show()
调用
plot_images_labels_prediction(mnist.validation.images,
mnist.validation.labels,[],0)
output
1.6 查看test
print('test images:', mnist.test.images.shape,
'labels:', mnist.test.labels.shape)
output
test images: (10000, 784) labels: (10000, 10)
可视化函数
plot_images_labels_prediction(mnist.test.images,
mnist.test.labels,[],0)
1.7 批次读取MINIST数据
mnist.train.next_batch
batch_images_xs, batch_labels_ys = \
mnist.train.next_batch(batch_size=100)
print(len(batch_images_xs),
len(batch_labels_ys))
output
100 100
2 Build Model
代码的由来可以参考这篇博客【TensorFlow】仿真Neural Network 3.1节
2.1 定义layer函数
def layer(output_dim,input_dim,inputs, activation=None):
W = tf.Variable(tf.random_normal([input_dim, output_dim]))
b = tf.Variable(tf.random_normal([1, output_dim]))
XWb = tf.matmul(inputs, W) + b
if activation is None:
outputs = XWb
else:
outputs = activation(XWb)
return outputs
keras和TensorFlow建立模型的区别，
Keras建立模型，只需要使用 model = Sequential（）建立线性堆叠模型，再使用model.add（）方法将各个神经网络加入模型即可。
TensorFlow 建立模型，必须自行定义 layer 函数（处理张量计算），然后使用 layer 函数构建 MLP。
2.2 用layer函数建立模型
# 建立输入层
x = tf.placeholder("float", [None, 784])
# 建立隐藏层h1
h1=layer(output_dim=256,input_dim=784,
inputs=x ,activation=tf.nn.relu)
# 建立输出层
y_predict=layer(output_dim=10,input_dim=256,
inputs=h1,activation=None)
Note：输入x的第一维设置为None是因为后续训练会传送很多数字图像，项数不固定，所以设置为None
2.3 定义训练方式
Keras 和 TensorFlow 定义训练方式不同
Keras：只需要使用model.compile 设置损失函数、优化器、并用 metrics 设置评估模型的方式
TensorFlow： 必须自行定义损失函数的公式、优化器和设置参数，并定义评估模型准确率的公式
# 建立训练数据 label 真实值的 placeholder
y_label = tf.placeholder("float", [None, 10])
# 定义 loss function
loss_function = tf.reduce_mean(
tf.nn.softmax_cross_entropy_with_logits
(logits=y_predict ,
labels=y_label))
# 选择 optimizer
optimizer = tf.train.AdamOptimizer(learning_rate=0.001) \
.minimize(loss_function)
2.4 定义模型的准确率的方式
# 计算每一项数据是否正確预测
# tf.equal,相等返回1，不等返回0
correct_prediction = tf.equal(tf.argmax(y_label , 1),
tf.argmax(y_predict, 1))
# 将計算预测正确结果，加权平均
# 先使用 tf.cast 转化为 float，再使用 tf.reduce_mean 将所有值平均
accuracy = tf.reduce_mean(tf.cast(correct_prediction, "float"))
3 Training process
3.1 参数设置
trainEpochs = 15
batchSize = 100
totalBatchs = int(mnist.train.num_examples/batchSize)
epoch_list=[];loss_list=[];accuracy_list=[]
from time import time
startTime=time()
sess = tf.Session() # 计算时间
sess.run(tf.global_variables_initializer())
3.2 Train
# for each epoch
for epoch in range(trainEpochs):
# for each mini-batch
for i in range(totalBatchs):
# 调用读mini-batch的函数
batch_x, batch_y = mnist.train.next_batch(batchSize)
# 训练优化器，最小化损失函数，输入x和y
sess.run(optimizer,feed_dict={x: batch_x,y_label: batch_y})
# 在验证集上计算精度和误差
loss,acc = sess.run([loss_function,accuracy],
feed_dict={x: mnist.validation.images,
y_label: mnist.validation.labels})
# 显示训练结果并存入列表
epoch_list.append(epoch)
loss_list.append(loss)
accuracy_list.append(acc)
print("Train Epoch:", '%02d' % (epoch+1), "Loss=", \
"{:.9f}".format(loss)," Accuracy=",acc)
# 计算训练的时间
duration =time()-startTime
print("Train Finished takes:",duration)
output
Train Epoch: 01 Loss= 6.564296246 Accuracy= 0.8392
Train Epoch: 02 Loss= 4.050883293 Accuracy= 0.8884
Train Epoch: 03 Loss= 3.251153946 Accuracy= 0.9018
Train Epoch: 04 Loss= 2.584557772 Accuracy= 0.9158
Train Epoch: 05 Loss= 2.280491114 Accuracy= 0.924
Train Epoch: 06 Loss= 2.051210642 Accuracy= 0.9308
Train Epoch: 07 Loss= 1.854478478 Accuracy= 0.934
Train Epoch: 08 Loss= 1.706961274 Accuracy= 0.9398
Train Epoch: 09 Loss= 1.629122496 Accuracy= 0.9428
Train Epoch: 10 Loss= 1.547713757 Accuracy= 0.9416
Train Epoch: 11 Loss= 1.541009307 Accuracy= 0.9394
Train Epoch: 12 Loss= 1.502972603 Accuracy= 0.9446
Train Epoch: 13 Loss= 1.435709476 Accuracy= 0.9448
Train Epoch: 14 Loss= 1.381304145 Accuracy= 0.9456
Train Epoch: 15 Loss= 1.344791651 Accuracy= 0.9464
Train Finished takes: 32.00210165977478
3.3 可视化精度和损失变化
先看看损失函数
%matplotlib inline
import matplotlib.pyplot as plt
fig = plt.gcf()
fig.set_size_inches(5,3)
plt.plot(epoch_list, loss_list, label = 'loss')
plt.ylabel('loss')
plt.xlabel('epoch')
plt.legend(['loss'])
plt.show()
output
再看看精度变化
plt.plot(epoch_list, accuracy_list,label="accuracy" )
fig = plt.gcf()
fig.set_size_inches(5,3)
plt.ylim(0.8,1) # 设置y轴的范围
plt.ylabel('accuracy')
plt.xlabel('epoch')
plt.legend()
plt.show()
output
4 评估模型的准确性
print("Accuracy:", sess.run(accuracy,
feed_dict={x: mnist.test.images,
y_label: mnist.test.labels}))
output
Accuracy: 0.9427
5 进行预测
prediction_result=sess.run(tf.argmax(y_predict,1),
feed_dict={x: mnist.test.images })
看看前10项
prediction_result[:10]
output
array([7, 2, 1, 0, 4, 1, 4, 9, 5, 9])
6 可视化预测结果
import matplotlib.pyplot as plt
import numpy as np
def plot_images_labels_prediction(images,labels,
prediction,idx,num=10):
fig = plt.gcf()
fig.set_size_inches(12, 14)
if num>25: num=25
for i in range(0, num):
ax=plt.subplot(5,5, 1+i)
ax.imshow(np.reshape(images[idx],(28, 28)),
cmap='binary')
title= "label=" +str(np.argmax(labels[idx]))
if len(prediction)>0:
title+=",predict="+str(prediction[idx])
ax.set_title(title,fontsize=10)
ax.set_xticks([]);ax.set_yticks([])
idx+=1
plt.show()
调用
plot_images_labels_prediction(mnist.test.images,
mnist.test.labels,
prediction_result,0)
output
7 查看softmax的结果
y_predict_Onehot=sess.run(y_predict,
feed_dict={x: mnist.test.images })
y_predict_Onehot[0]
output
array([ 18.89424 , -74.88019 , 15.298771, 44.221844, -77.21263 ,
10.695058, -48.48522 , 169.95691 , -10.677207, 33.074043],
dtype=float32)
8 找出预测出错的结果
for i in range(400):
if prediction_result[i]!=np.argmax(mnist.test.labels[i]):
print("i="+str(i)+
" label=",np.argmax(mnist.test.labels[i]),
"predict=",prediction_result[i])
output
i=18 label= 3 predict= 8
i=63 label= 3 predict= 8
i=66 label= 6 predict= 1
i=115 label= 4 predict= 9
i=126 label= 0 predict= 9
i=151 label= 9 predict= 8
i=158 label= 3 predict= 2
i=217 label= 6 predict= 5
i=241 label= 9 predict= 3
i=245 label= 3 predict= 6
i=247 label= 4 predict= 6
i=259 label= 6 predict= 0
i=289 label= 5 predict= 8
i=320 label= 9 predict= 1
i=321 label= 2 predict= 7
i=340 label= 5 predict= 3
i=341 label= 6 predict= 1
i=352 label= 5 predict= 3
9 MLP 1000
1-8节我们描述了 hidden layer 为256的 MLP。现在我们使 hidden layer 为1000，看看效果。
改一下 h1 的 output_dim 和 y_predict 的 output_dim=10 即可，其它不变
x = tf.placeholder("float", [None, 784])
h1=layer(output_dim=1000,input_dim=784,
inputs=x ,activation=tf.nn.relu)
y_predict=layer(output_dim=10,input_dim=1000,
inputs=h1,activation=None)
训练结果如下
Train Epoch: 01 Loss= 9.151745796 Accuracy= 0.8828
Train Epoch: 02 Loss= 5.984287262 Accuracy= 0.9124
Train Epoch: 03 Loss= 4.477479935 Accuracy= 0.9252
Train Epoch: 04 Loss= 3.777559757 Accuracy= 0.9342
Train Epoch: 05 Loss= 3.619176626 Accuracy= 0.9392
Train Epoch: 06 Loss= 3.215349674 Accuracy= 0.944
Train Epoch: 07 Loss= 2.910211325 Accuracy= 0.9484
Train Epoch: 08 Loss= 2.733655214 Accuracy= 0.9478
Train Epoch: 09 Loss= 2.763220787 Accuracy= 0.9516
Train Epoch: 10 Loss= 2.559421539 Accuracy= 0.9528
Train Epoch: 11 Loss= 2.659151077 Accuracy= 0.9496
Train Epoch: 12 Loss= 2.591297388 Accuracy= 0.9562
Train Epoch: 13 Loss= 2.547353506 Accuracy= 0.9552
Train Epoch: 14 Loss= 2.622963667 Accuracy= 0.9532
Train Epoch: 15 Loss= 2.446796179 Accuracy= 0.9584
Train Finished takes: 30.5201358795166
评估下准确率
print("Accuracy:", sess.run(accuracy,
feed_dict={x: mnist.test.images,
y_label: mnist.test.labels}))
output
Accuracy: 0.9575
比刚才好了一点点
10 MLP（two hidden layer）
区别在这，其它都一样
x = tf.placeholder("float", [None, 784])
h1=layer(output_dim=1000,input_dim=784,
inputs=x ,activation=tf.nn.relu)
h2=layer(output_dim=1000,input_dim=1000,
inputs=h1 ,activation=tf.nn.relu)
y_predict=layer(output_dim=10,input_dim=1000,
inputs=h2,activation=None)
训练过程如下
Train Epoch: 01 Loss= 116.442848206 Accuracy= 0.9206
Train Epoch: 02 Loss= 77.229240417 Accuracy= 0.9386
Train Epoch: 03 Loss= 56.488098145 Accuracy= 0.9526
Train Epoch: 04 Loss= 56.952060699 Accuracy= 0.9558
Train Epoch: 05 Loss= 52.448925018 Accuracy= 0.9558
Train Epoch: 06 Loss= 48.445960999 Accuracy= 0.962
Train Epoch: 07 Loss= 49.146423340 Accuracy= 0.9604
Train Epoch: 08 Loss= 55.970504761 Accuracy= 0.9562
Train Epoch: 09 Loss= 46.506732941 Accuracy= 0.9656
Train Epoch: 10 Loss= 41.754848480 Accuracy= 0.968
Train Epoch: 11 Loss= 45.703849792 Accuracy= 0.9654
Train Epoch: 12 Loss= 44.526569366 Accuracy= 0.9696
Train Epoch: 13 Loss= 36.788494110 Accuracy= 0.9722
Train Epoch: 14 Loss= 45.690227509 Accuracy= 0.9684
Train Epoch: 15 Loss= 42.659000397 Accuracy= 0.966
Train Finished takes: 28.71299958229065
评估下精度
print("Accuracy:", sess.run(accuracy,
feed_dict={x: mnist.test.images,
y_label: mnist.test.labels}))
结果为
Accuracy: 0.965
比刚才的又好一点，进一步提高的话需要用CNN 的方法了。
附录——Tensorflow计算网络的参数量
转载tensorflow统计网络参数量
最简单的方法如下：
def count2():
print np.sum([np.prod(v.get_shape().as_list()) for v in tf.trainable_variables()])
其它方法
def count1():
total_parameters = 0
for variable in tf.trainable_variables():
# shape is an array of tf.Dimension
shape = variable.get_shape()
# print(shape)
# print(len(shape))
variable_parameters = 1
for dim in shape:
# print(dim)
variable_parameters *= dim.value
# print(variable_parameters)
total_parameters += variable_parameters
print(total_parameters)
def get_nb_params_shape(shape):
'''
Computes the total number of params for a given shap.
Works for any number of shapes etc [D,F] or [W,H,C] computes D*F and W*H*C.
'''
nb_params = 1
for dim in shape:
nb_params = nb_params*int(dim)
return (nb_params)
def count3():
tot_nb_params = 0
for trainable_variable in tf.trainable_variables():
shape = trainable_variable.get_shape() # e.g [D,F] or [W,H,C]
current_nb_params = get_nb_params_shape(shape)
tot_nb_params = tot_nb_params + current_nb_params
print (tot_nb_params)
def count5():
total_parameters = 0
# iterating over all variables
for variable in tf.trainable_variables():
local_parameters = 1
shape = variable.get_shape() # getting shape of a variable
for i in shape:
local_parameters *= i.value # mutiplying dimension values
total_parameters += local_parameters
print(total_parameters)
def count6():
total_parameters = 0
for variable in tf.trainable_variables():
variable_parameters = 1
for dim in variable.get_shape():
variable_parameters *= dim.value
total_parameters += variable_parameters
print(total_parameters)
def count7():
from functools import reduce
from operator import mul
num_params = 0
for variable in tf.trainable_variables():
shape = variable.get_shape()
num_params += reduce(mul, [dim.value for dim in shape], 1)
print (num_params)
声明：代码源于《TensorFlow+Keras深度学习人工智能实践应用》 林大贵版，引用、转载请注明出处，谢谢，如果对书本感兴趣，买一本看看吧！！！
扫一扫
1万+
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
