神经网络算法：ResNet手写数字识别

ResNet手写数字识别

Kaiming He博士在ResNet论文中提出了这个思想,用于解决深层神经网络不好训练的问题.

上图是论文中关于传统神经网络不好识别的描述,可以看到.随着神经网络的深度增大,训练的错误率逐渐上升.于是他提出了利用残差学习的思想,就是机器不再学习整个分布,而是学习当前网络生成的分布与原始分布的差异.在CIFAR-10上的网络结构如下:

他使用如下的误差训练公式,y-x即为残差:

具体在CIFAR-10上的训练网络如下图最右侧所示,

这些跳过来的路径使得梯度反向传播时很少出现误差消失的情况.
我在mnist数据集上参考了几篇论文,实现了类似的神经网络,效果比较相似,但是mnist基本上不用做池化操作,因为毕竟每个样本的大小只有28*28.使用keras训练,最终准确率大概为0.9906,最终把模型保存为h5文件.下面直接上训练的代码.recognize是使用训练好的模型做手写数字识别的模块,本身和深度学习关系不大.网络结构大概是这个样子,keras导出的,使用

keras.utils.plot_model(self.model,to_file='mnist_res.png')

就可以把模型的结构可视化了.

'''用于训练mnist模型,生成h5模型文件'''
import tensorflow as tf
from tensorflow import keras
import recogonize
'''
识别的效果:
Epoch 1/5
 - 141s - loss: 0.2842 - acc: 0.9086 - val_loss: 0.0605 - val_acc: 0.9814
Epoch 2/5
 - 153s - loss: 0.0776 - acc: 0.9775 - val_loss: 0.0465 - val_acc: 0.9847
Epoch 3/5
 - 147s - loss: 0.0557 - acc: 0.9837 - val_loss: 0.0442 - val_acc: 0.9852
Epoch 4/5
 - 142s - loss: 0.0439 - acc: 0.9872 - val_loss: 0.0319 - val_acc: 0.9898
Epoch 5/5
 - 145s - loss: 0.0370 - acc: 0.9890 - val_loss: 0.0323 - val_acc: 0.9906
loss in test set: 0.032348819018113865
acc in test set: 0.9906
'''
class ResNet:
    def __init__(self,input_shape,classes,img_cols,img_rows):
        self.classes = classes
        self.img_cols = img_cols
        self.img_rows = img_rows
        self.input_shape = img_cols, img_rows, 1
        self.model = None
        (train_images, train_labels), (test_images, test_labels) = keras.datasets.mnist.load_data()
        self.train_images = train_images.reshape((train_images.shape[0], img_cols, img_rows, 1)).astype('float32')
        self.test_images = test_images.reshape((test_images.shape[0], img_cols, img_rows, 1)).astype('float32')
        self.train_images /= 255
        self.test_images /= 255
        self.train_labels = keras.utils.to_categorical(train_labels, classes)
        self.test_labels = keras.utils.to_categorical(test_labels, classes)

    #残差块,private
    def __res_block(self,x, channels, i):
        if i==1:
            xstrides = (1,1)
            x_add = x
        else:
            xstrides =(2,2)
            x_add = keras.layers.Conv2D(channels,
                                        kernel_size=3,
                                        activation='relu',
                                        padding='same',
                                        strides=(2,2)
                                        )(x)
        x = keras.layers.Conv2D(channels,
                                kernel_size=3,
                                activation='relu',
                                padding='same',
                                strides=xstrides)(x)
        x = keras.layers.Conv2D(channels,
                                kernel_size=3,
                                padding='same',
                                strides=(1,1))(x)

        x = keras.layers.add([x, x_add])
        x = keras.layers.Activation(keras.backend.relu)(x)
        return x

    def buildModel(self):
        firstin = keras.layers.Input(shape=self.input_shape)
        x = keras.layers.Conv2D(16,
                                kernel_size=7,
                                activation='relu',
                                input_shape=self.input_shape,
                                padding='same'
                                )(firstin)
        for i in range(2):
            x = self.__res_block(x,16,i)

        for i in range(2):
            x = self.__res_block(x,32,i)

        x = keras.layers.AveragePooling2D(pool_size=(7, 7))(x)
        x = keras.layers.Flatten()(x)
        x = keras.layers.Dense(10, activation='softmax')(x)

        self.model = keras.Model(inputs=firstin, outputs=x)
        keras.utils.plot_model(self.model,to_file='mnist_res.png')
        self.model.compile(loss='categorical_crossentropy',
                    optimizer='adadelta',
                    metrics=['accuracy']
                    )

    def train(self):
        self.model.fit(self.train_images,self.train_labels,
                batch_size=10,
                verbose=2,#精细化显示
                validation_data=(self.test_images,self.test_labels),
                epochs=5
                )
        self.model.save('mnist_res_model.h5')
        json_str = self.model.to_json()
        with open('mnist_res_model.json','w') as file:
            file.write(json_str)
        loss,acc = self.model.evaluate(self.test_images,self.test_labels,verbose=0)
        print("loss in test set:",loss)
        print("acc in test set:", acc)

    def test(self):
        recogonize.main(0)


def main(_):
    tr = ResNet((28,28,1),classes=10,img_cols=28,img_rows=28)
    tr.buildModel()
    tr.train()
    tr.test()

if __name__=="__main__":
    tf.app.run()