mnist数据集是机器学习领域中非常经典的一个数据集,由60000个训练样本和10000个测试样本组成,每个样本都是一张28*28像素的灰度手写数字图片。一共四个文件,训练集、训练集标签、测试集、测试集标签
- 实现mnist数据载入,可视化图形数字
- 实现数据预处理,图像数据维度转换与归一化,输出结果格式转换
- 计算模型在预测数据集的准确率
- 模型结构:两层隐藏层,每层有392个神经元
实现mnist数据载入,可视化图形数字
-
加载mnist数据集
from matplotlib import pyplot as plt (X_train, y_train), (X_test, y_test) = mnist.load_data() print(type(X_train), X_train.shape)
结果是
<class 'numpy.ndarray'> (60000, 28, 28)
-
可视化第一个图形数字
from matplotlib import pyplot as plt
# 可视化部分数据
img1 = X_train[0]
fig1 = plt.figure(figsize=(5, 5))
plt.imshow(img1)
plt.title(y_train[0])
plt.show()
结果是
从图形中可以看出来大概是数字5
输出维度print(img1.shape)
,结果的确是(28, 28)
每个样本都是一张28*28像素的灰度手写数字图片
实现数据预处理,图像数据维度转换与归一化,输出结果格式转换
-
图像数据维度转换
要把28*28=784的像素灰度转换成784列
feature_size = img1.shape[0] * img1.shape[1] X_train_format = X_train.reshape(X_train.shape[0], feature_size) print(X_train_format.shape) 结果是(60000, 784)
-
归一化
为了更快的进行mlp的迭代和求解
X_train_normal = X_train_format / 255 X_test_normal = X_test_format / 255
-
输出结果格式转换
from keras.utils import to_categorical y_train_format = to_categorical(y_train) y_test_format = to_categorical(y_test) print(y_train[0]) print(y_train_format[0]) 输出结果是5和[0. 0. 0. 0. 0. 1. 0. 0. 0. 0.]
模型结构搭建
-
建立模型结构
from keras.models import Sequential from keras.layers import Dense, Activation mlp = Sequential() mlp.add(Dense(units=392, activation='sigmoid', input_dim=feature_size)) mlp.add(Dense(units=392, activation='sigmoid')) mlp.add(Dense(units=10, activation='softmax')) mlp.summary()
输出结果是
-
模型配置
# 模型配置 mlp.compile(loss='categorical_crossentropy', optimizer='adam')
-
模型训练
# 模型训练 mlp.fit(X_train_normal, y_train_format, epochs=10)
-
模型评估
# 模型评估 # 训练集训练准确率 y_train_predict = mlp.predict_classes(X_train_normal) print(y_train_predict) accuracy_train = accuracy_score(y_train, y_train_predict) print(accuracy_train) # 测试集训练准确率 y_test_predict = mlp.predict_classes(X_test_normal) print(y_test_predict) accuracy_test = accuracy_score(y_test, y_test_predict) print(accuracy_test)
输出结果
可以看到这个模型训练的很好,训练集的准确率是0.99,测试集的准确率是0.98
-
直观查看模型的预测结果
# 测试集图形的可视化展示 img2 = X_test[100] fig2 = plt.figure(figsize=(3, 3)) plt.imshow(img2) plt.title(y_test_predict[100]) plt.show()
结果是
可以看到,预测的结果和照片中是一样的。
全部代码为
# 加载mnist数据集
from keras.datasets import mnist
from keras.utils import to_categorical
from keras.models import Sequential
from keras.layers import Dense, Activation
from sklearn.metrics import accuracy_score
from matplotlib import pyplot as plt
(X_train, y_train), (X_test, y_test) = mnist.load_data()
print(type(X_train), X_train.shape)
# 可视化部分数据
img1 = X_train[0]
fig1 = plt.figure(figsize=(3, 3))
plt.imshow(img1)
plt.title(y_train[0])
plt.show()
feature_size = img1.shape[0] * img1.shape[1]
X_train_format = X_train.reshape(X_train.shape[0], feature_size)
X_test_format = X_test.reshape(X_test.shape[0], feature_size)
print(X_test_format.shape)
X_train_normal = X_train_format / 255
X_test_normal = X_test_format / 255
y_train_format = to_categorical(y_train)
y_test_format = to_categorical(y_test)
print(y_train[0])
print(y_train_format[0])
# 建立模型结构
mlp = Sequential()
mlp.add(Dense(units=392, activation='sigmoid', input_dim=feature_size))
mlp.add(Dense(units=392, activation='sigmoid'))
mlp.add(Dense(units=10, activation='softmax'))
mlp.summary()
# 模型配置
mlp.compile(loss='categorical_crossentropy', optimizer='adam')
# 模型训练
mlp.fit(X_train_normal, y_train_format, epochs=10)
# 模型评估
# 训练集训练准确率
y_train_predict = mlp.predict_classes(X_train_normal)
print(y_train_predict)
accuracy_train = accuracy_score(y_train, y_train_predict)
print(accuracy_train)
# 测试集训练准确率
y_test_predict = mlp.predict_classes(X_test_normal)
print(y_test_predict)
accuracy_test = accuracy_score(y_test, y_test_predict)
print(accuracy_test)
# 测试集图形的可视化展示
img2 = X_test[100]
fig2 = plt.figure(figsize=(3, 3))
plt.imshow(img2)
plt.title(y_test_predict[100])
plt.show()
= accuracy_score(y_test, y_test_predict)
print(accuracy_test)
# 测试集图形的可视化展示
img2 = X_test[100]
fig2 = plt.figure(figsize=(3, 3))
plt.imshow(img2)
plt.title(y_test_predict[100])
plt.show()