使用MNIST数据集进行深度学习模型的训练，先看一下MNIST数据集有多少个样本呢：
读取方式：
import tensorflow as tf
import matplotlib.pyplot as plt
import numpy as np
from tensorflow.examples.tutorials.mnist import input_data
mnist_data_folder = "/MNIST_data" #指定数据集所在的位置
mnist = input_data.read_data_sets(mnist_data_folder,one_hot = True) #读取mnist数据集，其中one_hot是一种编码格式
#获取数据集的个数
train_nums = mnist.train.num_examples #训练集样本数
validation_nums = mnist.validation.num_examples #校正集样本数
test_nums = mnist.test.num_examples #测试集样本数
print("MNIST训练集样本个数：%d"%train_nums)
print("MNIST验证集样本个数：%d"%validation_nums)
print("MNIST测试集样本个数：%d"%test_nums)输出如下：
MNIST训练集样本个数：55000
MNIST验证集样本个数：5000
MNIST测试集样本个数：10000由此我们可以看到
数据集常常被划分为2部分（训练集、测试集）或三部分（训练集、测试集、验证集），各部分的作用：
训练集（Train set）:用来学习的一组例子，用于适应分类器的的参数（如神经网络的权重或阈值）。
验证集（Validation set）:一组用于调整分类器参数（即体系结构，而不是权重的示例）的示例，例如选择神经网络中隐层神经元的数量。
测试集（Test set）:一组用于评估完全指定分类器的的性能（泛化能力）的实例。
举个例子，训练集有点像模拟试卷，那么测试集就是期末大考的试卷，为了及时反馈学习状态调整学习方法，还可以设置月考，即验证集。
读取一个图片：
#获取数据值
train_data = mnist.train.images #所有的训练数据
val_data = mnist.validation.images
test_data = mnist.validation.images
print("训练集数据大小：",train_data.shape)
print("一幅图像大小：",train_data[2].shape)
print("一幅图像的列表表示：\n",train_data[2])显示如下：
训练集数据大小： (55000, 784)
一幅图像大小： (784,)
一幅图像的列表表示：
[0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0.02352941 0.76470596
0.9960785 1. 0.93725497 0.1137255 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0.02352941 0.6392157 0.9960785 0.9960785 0.6862745
0.21568629 0.01176471 0. 0. 0. 0.
0.09019608 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.47450984
0.9960785 0.9960785 0.5803922 0.03137255 0. 0.
0. 0. 0. 0.54901963 0.9294118 0.43921572
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0.16862746 0.9607844 0.9960785 0.9490197
0.01568628 0. 0. 0. 0. 0.
0.20784315 0.92549026 0.9960785 0.43921572 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0.6431373 0.9960785 0.9803922 0.2509804 0. 0.
0. 0. 0. 0. 0.6627451 0.9960785
0.97647065 0.1254902 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0.83921576 0.9960785
0.69803923 0. 0. 0.01176471 0.03529412 0.03529412
0.03529412 0.17254902 0.909804 0.9960785 0.64705884 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0.83921576 0.9960785 0.7254902 0.
0.21176472 0.7294118 0.9960785 0.9960785 0.9960785 0.9960785
0.9960785 0.9058824 0.16862746 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0.37647063 0.9960785 0.9803922 0.74509805 0.98823535 0.9960785
0.9960785 0.9960785 0.9960785 0.9960785 0.9960785 0.50980395
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0.49411768 0.9960785
0.9960785 0.9960785 0.9960785 0.7803922 0.46274513 0.5686275
0.9960785 0.9960785 0.9960785 0.30588236 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0.10980393 0.47058827 0.47058827 0.47058827
0.1254902 0.01176471 0. 0.47450984 0.9960785 0.9960785
0.76470596 0.01568628 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0.43137258 0.9960785 0.9960785 0.33333334 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.41960788
0.9960785 0.98823535 0.19215688 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0.8235295 0.9960785 0.7176471
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0.05882353 0.87843144 0.9960785 0.2784314 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0.27058825 0.9960785
0.97647065 0.20784315 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0.8470589 0.9960785 0.92549026 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0.8745099 0.9960785 0.74509805 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0.8745099 0.9960785
0.73333335 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0.8745099 0.9960785 0.97647065 0.49411768
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0.44705886 0.9333334 0.9960785 0.48627454 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. 0. 0.
0. 0. 0. 0. ]可以看到训练集的大小是55000*784，因为每幅图片是28*28=784。一幅图片就是784个像素点。
采用batch_size随机批量读取数据
#批量获取数据和标签，使用next_batch(batch_size)
#注意使用该方式的时候，数据是随机读取的，但在同一批次中，数据和标签是对应的
batch_size = 100
batch_xs,batch_ys = mnist.train.next_batch(batch_size)
testbatch_xs,testbatch_ys = mnist.test.next_batch(batch_size)
print("使用mnist.train.next_batch(batch_size)批量读取样本")
print("批量随机读取100个样本，数据集大小=",batch_xs.shape)
print("批量随机读取100个样本，标签集大小=",batch_ys.shape)
print("批量随机读取100个测试样本，数据集大小=",testbatch_xs.shape)
print("批量随机读取100个测试样本，标签集大小=",testbatch_ys.shape)显示如下
使用mnist.train.next_batch(batch_size)批量读取样本
批量随机读取100个样本，数据集大小= (100, 784)
批量随机读取100个样本，标签集大小= (100, 10)
批量随机读取100个测试样本，数据集大小= (100, 784)
批量随机读取100个测试样本，标签集大小= (100, 10)对图片进行显示
#显示图片
plt.figure()
for i in range(10):
im = batch_xs[i].reshape(28,28) #该批次的第一张图片
plt.imshow(im,cmap = plt.get_cmap('gray_r'))#白底黑字
plt.pause(0.5)#停留时间0.5s
plt.show()
如果我们要从所有的样本中挑出来一部分样本，可以使用以下代码：
#从mnist中选取一部分样本
from __future__ import division,print_function#python2中也能使用python3的函数
import tensorflow as tf
import matplotlib.pyplot as plt
import numpy as np #矩阵操作库
#Import MNIST data
from tensorflow.examples.tutorials.mnist import input_data
mnist = input_data.read_data_sets("MNIST_data/",one_hot = True)
#从训练集中选取前num个数据
def train_size(num):
print('Total Training Image in Dataset = '+
str(mnist.train.images.shape))
print('-----------------------------------')
x_train = mnist.train.images[:num,:]
print('x_train Examples Loaded = ' + str(x_train.shape))
y_train = mnist.train.labels[:num,:]
print('y_train Examples Loaded = ' + str(y_train.shape))
print('')
return x_train,y_train
#从测试集中选取前num个数据
def test_size(num):
print('Total Test Examples in Dataset = '+
str(mnist.test.images.shape))
print('-----------------------------------')
x_test = mnist.test.images[:num,:]
print('x_test Examples Loaded =' + str(x_test.shape))
y_test = mnist.test.labels[:num,:]
print('y_test Examples Loaded =' + str(y_test.shape))
return x_test,y_test
#显示训练集中的第num个数据，以图片的形式显示
def display_digit(num):
print(y_train[num])
label = y_train[num].argmax(axis = 0)#返回最大值索引值，也即是该数字大小
images = x_train[num].reshape([28,28])
plt.title('Example: %d Label: %d'%(num ,label))#显示随机读取的图片的编号
plt.imshow(images,cmap = plt.get_cmap('gray_r'))
plt.show()
#显示训练集中第start~stop的数据，每一列代表一个样本
def display_mult_flat(start,stop):
images = x_train[start].reshape([1,784])
for i in range(start+1,stop): #进行拼接
images = np.concatenate((images,
x_train[i].reshape([1,784])))
print(images.shape)
plt.imshow(images, cmap = plt.get_cmap('gray_r'))
plt.show()
x_train,y_train = train_size(55000)#训练集数据选取55000个
display_digit(np.random.randint(0,x_train.shape[0]))#随机读取一张照片
display_mult_flat(0,400)
1120
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
