基于Keras的LetNet深度神经网络学习

卷积神经网络

上一章对于MNIST手写体识别使用2层神经网络，accuracy = 92.2%。然而，这样的策略并未提取出空间结构和图像关系信息。其中，以下代码把每个位图表示的手写体数组转换成扁平向量，导致空间局部性消失。

# X_train是60000行28*28的数据，变形为60000*784
X_train = X_train.reshape(60000, 784)
X_test = X_test.reshape(10000, 784)

卷积神经网络保留了空间信息，因此可以更好地使用于图像分类问题。如科学研究表明，我们的视觉基于多个皮质层，每层识别越来越多的结构性信息。首先，我们看到的是很多单个的像素；然后从这些像素中，我们识别出几何组成；再然后……这样越来越多的复杂的元素，最终物体、面部、人体躯干、动物被识别出来。

深度卷积神经网络——LeNet

用以训练MNIST手写字符集的识别，它对简单几何变换和扭转具有很好的鲁棒性。这里的关键点是让较低的网络层交替进行卷积和最大池化运算。
卷积操作基于仔细甄选的局部感受野，它们在多个特征平面共享权值。之后，更高的全连接网络层基于传统的多层感知机，它们包含隐藏层并将softmax作为输出层。
LeNet主要思路，第一层（20个卷积层），第二层（Max池化层），第三层（50个卷积层），第四层（Max池化层），第五层（全连接层），第六层（全连接输出层），以下就是程序：

# import the necessary packages
from keras import backend as K
from keras.models import Sequential
from keras.layers.convolutional import Conv2D
from keras.layers.convolutional import MaxPooling2D
from keras.layers.core import Activation
from keras.layers.core import Flatten
from keras.layers.core import Dense
from keras.datasets import mnist
from keras.utils import np_utils
from keras.optimizers import SGD, RMSprop, Adam
import numpy as np

import matplotlib.pyplot as plt

np.random.seed(1671)  # for reproducibility


# define the convnet
class LeNet:
    @staticmethod
    def build(input_shape, classes):
        model = Sequential()
        # CONV => RELU => POOL
        #进行pad零填充 padding = 'same' ,kernel_size 声明了二维卷积窗的宽度和高度, 20个卷积滤波器
        # 因为使用零填充，所以输出与输入一致，输出28*28*1
        model.add(Conv2D(20, kernel_size=(5, 5), padding="same",
                         input_shape=input_shape))
        model.add(Activation("relu"))
        # stride=(2,2) 表示将使图片在两个维度上变成原来的一半，所以输出为14*14*1
        model.add(MaxPooling2D(pool_size=(2, 2), strides=2))
        # CONV => RELU => POOL
        #50个卷积滤波器，在更深的网络层增加滤波器数目使深度学习中一个普遍采用的技术，输入于输出一致
        model.add(Conv2D(50, kernel_size=(5, 5), padding="same"))
        model.add(Activation("relu"))
        model.add(MaxPooling2D(pool_size=(2, 2), strides=2))
        # Flatten => RELU layers
        #相当标准和扁平的全连接网络， 包含500个神经元， 其后是具有10个类别的softmax分类器
        model.add(Flatten())
        model.add(Dense(500))
        model.add(Activation("relu"))

        # a softmax classifier
        model.add(Dense(classes))
        model.add(Activation("softmax"))

        return model


# network and training
NB_EPOCH = 9
BATCH_SIZE = 128
VERBOSE = 1
OPTIMIZER = Adam()
VALIDATION_SPLIT = 0.2

IMG_ROWS, IMG_COLS = 28, 28  # input image dimensions
NB_CLASSES = 10  # number of outputs = number of digits
INPUT_SHAPE = (IMG_ROWS, IMG_COLS, 1)

# data: shuffled and split between train and test sets
(X_train, y_train), (X_test, y_test) = mnist.load_data()
K.image_data_format() == "channels_first"

# consider them as float and normalize
X_train = X_train.astype('float32')
X_test = X_test.astype('float32')
X_train /= 255
X_test /= 255

# we need a 60K x [28*28*1] shape as input to the CONVNET
X_train = X_train.reshape(-1, 28, 28, 1)  # normalize
X_test = X_test.reshape(-1, 28, 28, 1)      # normalize

print(X_train.shape[0], 'train samples')
print(X_test.shape[0], 'test samples')

# convert class vectors to binary class matrices
y_train = np_utils.to_categorical(y_train, num_classes=NB_CLASSES)
y_test = np_utils.to_categorical(y_test, num_classes=NB_CLASSES)

# initialize the optimizer and model
model = LeNet.build(input_shape=INPUT_SHAPE, classes=NB_CLASSES)
model.compile(loss="categorical_crossentropy", optimizer=OPTIMIZER,
              metrics=["accuracy"])

history = model.fit(X_train, y_train,
                    batch_size=BATCH_SIZE, epochs=NB_EPOCH,
                    verbose=VERBOSE, validation_split=VALIDATION_SPLIT)

score = model.evaluate(X_test, y_test, verbose=VERBOSE)
print("\nTest score:", score[0])
print('Test accuracy:', score[1])

# list all data in history
print(history.history.keys())
# summarize history for accuracy
# if val 'accuracy' is wrong, please use 'acc' replace it.
# if val 'history' is wrong, please use 'hist' replace it.
plt.plot(history.history['accuracy'])
plt.plot(history.history['val_accuracy'])
plt.title('model accuracy')
plt.ylabel('accuracy')
plt.xlabel('epoch')
plt.legend(['train', 'test'], loc='upper left')
plt.show()
# summarize history for loss
plt.plot(history.history['loss'])
plt.plot(history.history['val_loss'])
plt.title('model loss')
plt.ylabel('loss')
plt.xlabel('epoch')
plt.legend(['train', 'test'], loc='upper left')
plt.show()

总结

经过LeNet深度神经网络，MNIST手写体识别准确性可达到99.1%，相当于100张图片只有1张识别错误。顺便包括，训练和测试中的损失以及准确率曲线图。至此MNIST数据集完结，之后为CIFAR-10的数据集识别。