keras复现ResNet_18

原来复现了ResNet_50。今天再复现ResNet_18的时候才发现原来ResNet_50的复现并不正确。错误地方是论文中的卷积核大小都是（3,3）。而使用keras复现的时候由于卷积层输入输出大小的计算公式不熟悉，误打误撞，虽然复现后的模型plot_model、甚至是model.summary()出来的结果都是一模一样的，但是却不是真正的ResNet_50。今天先正确复现ResNet_18，改天抽空再复现ResNet_50。
ResNet论文获取连接：《Deep Residual Learning for Image Recognition》
论文中展示的模型表格是这样的：

其中论文中很重要的一段话是：
“Our plain baselines (Fig. 3, middle) are mainly inspired by the philosophy of VGG nets [40] (Fig. 3, left). The convolutional layers mostly have 3×3 filters and follow two simple design rules: (i) for the same output feature map size, the layers have the same number of filters; and (ii) if the feature map size is halved, the number of filters is doubled so as to preserve the time complexity per layer. We perform downsampling directly by convolutional layers that have a stride of 2. The network ends with a global average pooling layer and a 1000-way fully-connected layer with softmax. The total number of weighted layers is 34 in Fig. 3 (middle)”
在github上翻译的中文是：
“我们简单网络的基准（图3，中间）主要受到VGG网络[40]（图3，左图）的哲学启发。卷积层主要有3×3的滤波器，并遵循两个简单的设计规则：（i）对于相同的输出特征图尺寸，层具有相同数量的滤波器；（ii）如果特征图尺寸减半，则滤波器数量加倍，以便保持每层的时间复杂度。我们通过步长为2的卷积层直接执行下采样。网络以全局平均池化层和具有softmax的1000维全连接层结束。图3（中间）的加权层总数为34。”
这段话等会在代码中显示出来或许会更形象一点。
在图3中右边的图中，既有实线，也有虚线：

关于这个实线和虚线，分别代表什么意思？论文中也作了说明：
“Based on the above plain network, we insert shortcut connections (Fig. 3, right) which turn the network into its counterpart residual version. The identity shortcuts (Eqn.(1)) can be directly used when the input and output are of the same dimensions (solid line shortcuts in Fig. 3). When the dimensions increase (dotted line shortcuts in Fig. 3), we consider two options: (A) The shortcut still performs identity mapping, with extra zero entries padded
for increasing dimensions. This option introduces no extra parameter; (B) The projection shortcut in Eqn.(2) is used to match dimensions (done by 1×1 convolutions). For both options, when the shortcuts go across feature maps of two sizes, they are performed with a stride of 2.”
在github上翻译的中文是：
“基于上述的简单网络，我们插入快捷连接（图3，右），将网络转换为其对应的残差版本。当输入和输出具有相同的维度时（图3中的实线快捷连接）时，可以直接使用恒等快捷连接（方程（1））。当维度增加（图3中的虚线快捷连接）时，我们考虑两个选项：（A）快捷连接仍然执行恒等映射，额外填充零输入以增加维度。此选项不会引入额外的参数；（B）方程（2）中的投影快捷连接用于匹配维度（由1×1卷积完成）。对于这两个选项，当快捷连接跨越两种尺寸的特征图时，它们执行时步长为2。”

根据这两幅图和这两段话，我的理解是：在残差连接中，如果是从（56, 56, 64）到（56, 56, 64）的残差连接，卷积步长为1；如果是从（56, 56, 64）到（28, 28, 128）的残差连接，卷积步长为2，即strides=(2,2)。这也是对图中虚线中128后面多了 “/2”的理解。
还有就是：“快捷连接仍然执行恒等映射，额外填充零输入以增加维度。此选项不会引入额外的参数”指的应该是卷积中的padding=“same”；
“方程（2）中的投影快捷连接用于匹配维度（由1×1卷积完成）”应该指的是卷积操作时kernel_size =（1,1）。
关于方程（1）和方程（2）可以去看看论文。（看论文看到这种公式就很头疼 || >_< ||）。

好了，话不多说，看代码：

# -*- coding: utf-8 -*-
"""
Created on Mon May 11 13:45:20 2020

@author: Administrator
"""
from keras import layers
from keras import Model


filters = 64
width = 224
channals = 3
nb_classes = 2

input_shape = (width, width, channals)

input_1 = layers.Input(shape=input_shape)

def conv1():    
    zpad = layers.ZeroPadding2D(padding=(3,3), data_format='channels_last')(input_1)
    '''
    加ZeroPadding2D是因为通过keras的API导出的ResNet_50模型中有ZeroPadding2D层，而ResNet_18和ResNet_50的开头都是一样的。
    '''
    con = layers.Conv2D(filters=64, kernel_size=(7,7),strides=(2,2), padding='valid')(zpad)
    bn = layers.BatchNormalization()(con)
    ac = layers.Activation('relu')(bn)
    zpad = layers.ZeroPadding2D()(ac)
    mp = layers.MaxPooling2D(pool_size=(3,3), strides=(2,2))(zpad)
    return mp

##########################################################################
def conv2_x(cat):
	'''
	参数cat是指前一个模块，接下来搭建的层接到这个cat模块上。
	在第一张图中可以看出conv2_x是两个有两层卷积的模块。所以这里搭建两个卷积层。
	'''
    con = layers.Conv2D(filters=64, kernel_size=(3,3),strides=(1,1), padding='same')(cat)
    bn = layers.BatchNormalization()(con)
    ac = layers.Activation('relu')(bn)
    
    con = layers.Conv2D(filters=64, kernel_size=(3,3),strides=(1,1), padding='same')(ac)
    bn = layers.BatchNormalization()(con)
    
    conv2_x_add = layers.add([bn, cat])
   
    ac = layers.Activation('relu')(conv2_x_add)
    
    return ac

def conv3_x(cat, strides = 1):
	'''
	参数cat是指前一个模块，接下来搭建的层接到这个cat模块上。
	在第一张图中可以看出conv3_x是两个有两层卷积的模块。所以这里搭建两个卷积层。
	与此同时，还要判断当前的两个卷积层的上一个残差连接模块的维度是64还是128？
	通过strides参数来判断。
	strides = 1说明上一个残差连接模块的维度是128。
	strides = 2说明上一个残差连接模块的维度是64。需要把上个残差模块的特征图进行卷积降维到28。也就是特征图尺寸减半，滤波器数量加倍。
	'''
    if strides == 2:
        con = layers.Conv2D(filters=128, kernel_size=(3,3),strides=(2,2), padding='same')(cat)#特征图尺寸减半，滤波器数量加倍。
    else:
        con = layers.Conv2D(filters=128, kernel_size=(3,3),strides=(1,1), padding='same')(cat)#相同的输出特征图，层具有相同数量的滤波器
    
    bn = layers.BatchNormalization()(con)
    ac = layers.Activation('relu')(bn)
    con = layers.Conv2D(filters=128, kernel_size=(3,3),strides=(1,1), padding='same')(ac)
    bn1 = layers.BatchNormalization()(con)

    if strides == 2:
        con = layers.Conv2D(filters=128, kernel_size=(3,3),strides=(2,2), padding='same')(cat)#特征图尺寸减半，滤波器数量加倍。
        bn2 = layers.BatchNormalization()(con)
        conv2_x_add = layers.add([bn1, bn2])
    else:
        conv2_x_add = layers.add([bn1, cat])#相同的输出特征图，层具有相同数量的滤波器
    
    ac = layers.Activation('relu')(conv2_x_add)
    return ac

def conv4_x(cat, strides = 1):
	'''
	参数说明与conv3_x一样，这里仅改变filters的个数
	'''
    if strides == 2:
        con = layers.Conv2D(filters=256, kernel_size=(3,3),strides=(2,2), padding='same')(cat)#特征图尺寸减半，滤波器数量加倍。
    else:
        con = layers.Conv2D(filters=256, kernel_size=(3,3),strides=(1,1), padding='same')(cat)#相同的输出特征图，层具有相同数量的滤波器
    
    bn = layers.BatchNormalization()(con)
    ac = layers.Activation('relu')(bn)
    con = layers.Conv2D(filters=256, kernel_size=(3,3),strides=(1,1), padding='same')(ac)
    bn1 = layers.BatchNormalization()(con)

    if strides == 2:
        con = layers.Conv2D(filters=256, kernel_size=(3,3),strides=(2,2), padding='same')(cat)#特征图尺寸减半，滤波器数量加倍。
        bn2 = layers.BatchNormalization()(con)
        conv2_x_add = layers.add([bn1, bn2])
    else:
        conv2_x_add = layers.add([bn1, cat])#相同的输出特征图，层具有相同数量的滤波器
    
    ac = layers.Activation('relu')(conv2_x_add)
    return ac


def conv5_x(cat, strides = 1):
	'''
	参数说明与conv3_x一样，这里仅改变filters的个数
	'''
    if strides == 2:
        con = layers.Conv2D(filters=512, kernel_size=(3,3),strides=(2,2), padding='same')(cat)#特征图尺寸减半，滤波器数量加倍。
    else:
        con = layers.Conv2D(filters=512, kernel_size=(3,3),strides=(1,1), padding='same')(cat)#相同的输出特征图，层具有相同数量的滤波器
    
    bn = layers.BatchNormalization()(con)
    ac = layers.Activation('relu')(bn)
    con = layers.Conv2D(filters=512, kernel_size=(3,3),strides=(1,1), padding='same')(ac)
    bn1 = layers.BatchNormalization()(con)

    if strides == 2:
        con = layers.Conv2D(filters=512, kernel_size=(3,3),strides=(2,2), padding='same')(cat)#特征图尺寸减半，滤波器数量加倍。
        bn2 = layers.BatchNormalization()(con)
        conv2_x_add = layers.add([bn1, bn2])
    else:
        conv2_x_add = layers.add([bn1, cat])#相同的输出特征图，层具有相同数量的滤波器
    
    ac = layers.Activation('relu')(conv2_x_add)
    return ac


##########################################################################
con1 = conv1()
##########################################################################
con2 = conv2_x(con1)
con2 = conv2_x(con2)
'''
卷积运算过后特征图的大小（维度）计算公式： output_shape = (input_shape  - kernel_size + 2*p)/strides + 1

con = layers.Conv2D(filters=128, kernel_size=(3,3),strides=(1,1), padding='valid')(con2)#output=(54, 54, 128). 54 = (56-3+2*0)/1 + 1
con = layers.Conv2D(filters=128, kernel_size=(3,3),strides=(2,2), padding='valid')(con2)#output=(27, 27, 128). 54 = (56-3+2*0)/2 + 1。除不尽时向下取整

con = layers.Conv2D(filters=128, kernel_size=(3,3),strides=(1,1), padding='same')(con2)#output=(56, 56, 128). 56 = (56-3+2*1)/1 + 1
con = layers.Conv2D(filters=128, kernel_size=(3,3),strides=(2,2), padding='same')(con2)#output=(28, 28, 128). 28 = (56-3+2*1)/2 + 1。

con = layers.Conv2D(filters=128, kernel_size=(1,1),strides=(1,1), padding='valid')(con2)#output=(56, 56, 128). 56 = (56-1+2*0)/1 + 1。相当于逐点卷积
con = layers.Conv2D(filters=128, kernel_size=(1,1),strides=(2,2), padding='valid')(con2)#output=(28, 28, 128). 28 = (56-1+2*0)/2 + 1。除不尽时向下取整

con = layers.Conv2D(filters=128, kernel_size=(1,1),strides=(1,1), padding='same')(con2)#output=(56, 56, 128). 56 = (56-1+2*1)/1 + 1。相当于逐点卷积
con = layers.Conv2D(filters=128, kernel_size=(1,1),strides=(2,2), padding='same')(con2)#output=(28, 28, 128). 29 = (56-1+2*1)/2 + 1。因为same

model = Model(inputs=input_1, outputs=con)
model.summary()
通过上述的计算，发现有两种卷积设置可以达到输出大小为28。那ResNet的作者凯明同学是用了哪种方法呢?
这里选择使用kernel_size=(3,3),strides=(2,2), padding='same'。
因为第1幅图中显示的卷积核大小都是(3,3)并且第2幅图中有个 "/2"。

'''
##########################################################################
con3 = conv3_x(con2,strides=2)
con3 = conv3_x(con3,strides=1)
##########################################################################
con4 = conv4_x(con3, strides=2)
con4 = conv4_x(con4, strides=1)

##########################################################################
con5 = conv5_x(con4, strides=2)
con5 = conv5_x(con5, strides=1)
##########################################################################
avg = layers.AveragePooling2D(pool_size=(7,7), padding='same')(con5)
flatten = layers.Flatten()(avg)
if nb_classes == 1:
    dense = layers.Dense(units=nb_classes, activation='sigmoid')(flatten)
else:
    dense = layers.Dense(units=nb_classes, activation='softmax')(flatten)

model = Model(inputs=[input_1], outputs=[dense])


model.summary()

model.summary()的结果为：

Model: "model_5"
__________________________________________________________________________________________________
Layer (type)                    Output Shape         Param #     Connected to                     
==================================================================================================
input_3 (InputLayer)            (None, 224, 224, 3)  0                                            
__________________________________________________________________________________________________
zero_padding2d_5 (ZeroPadding2D (None, 230, 230, 3)  0           input_3[0][0]                    
__________________________________________________________________________________________________
conv2d_28 (Conv2D)              (None, 112, 112, 64) 9472        zero_padding2d_5[0][0]           
__________________________________________________________________________________________________
batch_normalization_26 (BatchNo (None, 112, 112, 64) 256         conv2d_28[0][0]                  
__________________________________________________________________________________________________
activation_23 (Activation)      (None, 112, 112, 64) 0           batch_normalization_26[0][0]     
__________________________________________________________________________________________________
zero_padding2d_6 (ZeroPadding2D (None, 114, 114, 64) 0           activation_23[0][0]              
__________________________________________________________________________________________________
max_pooling2d_3 (MaxPooling2D)  (None, 56, 56, 64)   0           zero_padding2d_6[0][0]           
__________________________________________________________________________________________________
conv2d_29 (Conv2D)              (None, 56, 56, 64)   36928       max_pooling2d_3[0][0]            
__________________________________________________________________________________________________
batch_normalization_27 (BatchNo (None, 56, 56, 64)   256         conv2d_29[0][0]                  
__________________________________________________________________________________________________
activation_24 (Activation)      (None, 56, 56, 64)   0           batch_normalization_27[0][0]     
__________________________________________________________________________________________________
conv2d_30 (Conv2D)              (None, 56, 56, 64)   36928       activation_24[0][0]              
__________________________________________________________________________________________________
batch_normalization_28 (BatchNo (None, 56, 56, 64)   256         conv2d_30[0][0]                  
__________________________________________________________________________________________________
add_11 (Add)                    (None, 56, 56, 64)   0           batch_normalization_28[0][0]     
                                                                 max_pooling2d_3[0][0]            
__________________________________________________________________________________________________
activation_25 (Activation)      (None, 56, 56, 64)   0           add_11[0][0]                     
__________________________________________________________________________________________________
conv2d_31 (Conv2D)              (None, 56, 56, 64)   36928       activation_25[0][0]              
__________________________________________________________________________________________________
batch_normalization_29 (BatchNo (None, 56, 56, 64)   256         conv2d_31[0][0]                  
__________________________________________________________________________________________________
activation_26 (Activation)      (None, 56, 56, 64)   0           batch_normalization_29[0][0]     
__________________________________________________________________________________________________
conv2d_32 (Conv2D)              (None, 56, 56, 64)   36928       activation_26[0][0]              
__________________________________________________________________________________________________
batch_normalization_30 (BatchNo (None, 56, 56, 64)   256         conv2d_32[0][0]                  
__________________________________________________________________________________________________
add_12 (Add)                    (None, 56, 56, 64)   0           batch_normalization_30[0][0]     
                                                                 activation_25[0][0]              
__________________________________________________________________________________________________
activation_27 (Activation)      (None, 56, 56, 64)   0           add_12[0][0]                     
__________________________________________________________________________________________________
conv2d_33 (Conv2D)              (None, 28, 28, 128)  73856       activation_27[0][0]              
__________________________________________________________________________________________________
batch_normalization_31 (BatchNo (None, 28, 28, 128)  512         conv2d_33[0][0]                  
__________________________________________________________________________________________________
activation_28 (Activation)      (None, 28, 28, 128)  0           batch_normalization_31[0][0]     
__________________________________________________________________________________________________
conv2d_34 (Conv2D)              (None, 28, 28, 128)  147584      activation_28[0][0]              
__________________________________________________________________________________________________
conv2d_35 (Conv2D)              (None, 28, 28, 128)  73856       activation_27[0][0]              
__________________________________________________________________________________________________
batch_normalization_32 (BatchNo (None, 28, 28, 128)  512         conv2d_34[0][0]                  
__________________________________________________________________________________________________
batch_normalization_33 (BatchNo (None, 28, 28, 128)  512         conv2d_35[0][0]                  
__________________________________________________________________________________________________
add_13 (Add)                    (None, 28, 28, 128)  0           batch_normalization_32[0][0]     
                                                                 batch_normalization_33[0][0]     
__________________________________________________________________________________________________
activation_29 (Activation)      (None, 28, 28, 128)  0           add_13[0][0]                     
__________________________________________________________________________________________________
conv2d_36 (Conv2D)              (None, 28, 28, 128)  147584      activation_29[0][0]              
__________________________________________________________________________________________________
batch_normalization_34 (BatchNo (None, 28, 28, 128)  512         conv2d_36[0][0]                  
__________________________________________________________________________________________________
activation_30 (Activation)      (None, 28, 28, 128)  0           batch_normalization_34[0][0]     
__________________________________________________________________________________________________
conv2d_37 (Conv2D)              (None, 28, 28, 128)  147584      activation_30[0][0]              
__________________________________________________________________________________________________
batch_normalization_35 (BatchNo (None, 28, 28, 128)  512         conv2d_37[0][0]                  
__________________________________________________________________________________________________
add_14 (Add)                    (None, 28, 28, 128)  0           batch_normalization_35[0][0]     
                                                                 activation_29[0][0]              
__________________________________________________________________________________________________
activation_31 (Activation)      (None, 28, 28, 128)  0           add_14[0][0]                     
__________________________________________________________________________________________________
conv2d_38 (Conv2D)              (None, 14, 14, 256)  295168      activation_31[0][0]              
__________________________________________________________________________________________________
batch_normalization_36 (BatchNo (None, 14, 14, 256)  1024        conv2d_38[0][0]                  
__________________________________________________________________________________________________
activation_32 (Activation)      (None, 14, 14, 256)  0           batch_normalization_36[0][0]     
__________________________________________________________________________________________________
conv2d_39 (Conv2D)              (None, 14, 14, 256)  590080      activation_32[0][0]              
__________________________________________________________________________________________________
conv2d_40 (Conv2D)              (None, 14, 14, 256)  295168      activation_31[0][0]              
__________________________________________________________________________________________________
batch_normalization_37 (BatchNo (None, 14, 14, 256)  1024        conv2d_39[0][0]                  
__________________________________________________________________________________________________
batch_normalization_38 (BatchNo (None, 14, 14, 256)  1024        conv2d_40[0][0]                  
__________________________________________________________________________________________________
add_15 (Add)                    (None, 14, 14, 256)  0           batch_normalization_37[0][0]     
                                                                 batch_normalization_38[0][0]     
__________________________________________________________________________________________________
activation_33 (Activation)      (None, 14, 14, 256)  0           add_15[0][0]                     
__________________________________________________________________________________________________
conv2d_41 (Conv2D)              (None, 14, 14, 256)  590080      activation_33[0][0]              
__________________________________________________________________________________________________
batch_normalization_39 (BatchNo (None, 14, 14, 256)  1024        conv2d_41[0][0]                  
__________________________________________________________________________________________________
activation_34 (Activation)      (None, 14, 14, 256)  0           batch_normalization_39[0][0]     
__________________________________________________________________________________________________
conv2d_42 (Conv2D)              (None, 14, 14, 256)  590080      activation_34[0][0]              
__________________________________________________________________________________________________
batch_normalization_40 (BatchNo (None, 14, 14, 256)  1024        conv2d_42[0][0]                  
__________________________________________________________________________________________________
add_16 (Add)                    (None, 14, 14, 256)  0           batch_normalization_40[0][0]     
                                                                 activation_33[0][0]              
__________________________________________________________________________________________________
activation_35 (Activation)      (None, 14, 14, 256)  0           add_16[0][0]                     
__________________________________________________________________________________________________
conv2d_43 (Conv2D)              (None, 7, 7, 512)    1180160     activation_35[0][0]              
__________________________________________________________________________________________________
batch_normalization_41 (BatchNo (None, 7, 7, 512)    2048        conv2d_43[0][0]                  
__________________________________________________________________________________________________
activation_36 (Activation)      (None, 7, 7, 512)    0           batch_normalization_41[0][0]     
__________________________________________________________________________________________________
conv2d_44 (Conv2D)              (None, 7, 7, 512)    2359808     activation_36[0][0]              
__________________________________________________________________________________________________
conv2d_45 (Conv2D)              (None, 7, 7, 512)    1180160     activation_35[0][0]              
__________________________________________________________________________________________________
batch_normalization_42 (BatchNo (None, 7, 7, 512)    2048        conv2d_44[0][0]                  
__________________________________________________________________________________________________
batch_normalization_43 (BatchNo (None, 7, 7, 512)    2048        conv2d_45[0][0]                  
__________________________________________________________________________________________________
add_17 (Add)                    (None, 7, 7, 512)    0           batch_normalization_42[0][0]     
                                                                 batch_normalization_43[0][0]     
__________________________________________________________________________________________________
activation_37 (Activation)      (None, 7, 7, 512)    0           add_17[0][0]                     
__________________________________________________________________________________________________
conv2d_46 (Conv2D)              (None, 7, 7, 512)    2359808     activation_37[0][0]              
__________________________________________________________________________________________________
batch_normalization_44 (BatchNo (None, 7, 7, 512)    2048        conv2d_46[0][0]                  
__________________________________________________________________________________________________
activation_38 (Activation)      (None, 7, 7, 512)    0           batch_normalization_44[0][0]     
__________________________________________________________________________________________________
conv2d_47 (Conv2D)              (None, 7, 7, 512)    2359808     activation_38[0][0]              
__________________________________________________________________________________________________
batch_normalization_45 (BatchNo (None, 7, 7, 512)    2048        conv2d_47[0][0]                  
__________________________________________________________________________________________________
add_18 (Add)                    (None, 7, 7, 512)    0           batch_normalization_45[0][0]     
                                                                 activation_37[0][0]              
__________________________________________________________________________________________________
activation_39 (Activation)      (None, 7, 7, 512)    0           add_18[0][0]                     
__________________________________________________________________________________________________
average_pooling2d_2 (AveragePoo (None, 1, 1, 512)    0           activation_39[0][0]              
__________________________________________________________________________________________________
flatten_2 (Flatten)             (None, 512)          0           average_pooling2d_2[0][0]        
__________________________________________________________________________________________________
dense_2 (Dense)                 (None, 2)            1026        flatten_2[0][0]                  
==================================================================================================
Total params: 12,568,194
Trainable params: 12,558,594
Non-trainable params: 9,600
__________________________________________________________________________________________________

画出来的样子是：

（这是没有加flatten层的图，为了训练自己的数据加上flatten层。）
在这个模型上训练自己的数据集（125×125×3），averagepooling中的pool_size=(4,4)。是可以训练的。（restart kernel没有训练历史了，就不贴了）。

总结：总算是真正看懂第二幅图中实线和虚线的含义了。只要把ResNet_18复现出来，剩下的ResNet_34和ResNet_50就按照第一幅图中的框架，继续搭，复现应该会很快了。
另外在简书上也值得参考的文章：基于keras的Resnet
基于pytorch的复现：pytorch之ResNet18（对cifar10数据进行分类准确度达到94%）