mxnet，gluon学习笔记3——主干网络实现代码对比：AlexNet,VGG,NIN,googLeNet,ResNet,DesNet

参考李沐的mxnet学习资料。

注，为了实际应用，部分dim参数我调整减小，但网络结构主体思想不变。

from mxnet.gluon import nn
from mxnet import nd

1. LeNet

net = nn.Sequential()
net.add(nn.Conv2D(channels=6, kernel_size=5, activation='sigmoid'),
        nn.MaxPool2D(pool_size=4, strides=2),
        nn.Conv2D(channels=16, kernel_size=5, activation='sigmoid'),
        nn.MaxPool2D(pool_size=2, strides=2),
        nn.Dense(120, activation='sigmoid'),
        nn.Dense(84, activation='sigmoid'),
        nn.Dense(10))

说明

• Dense()会默认将（批量大小，通道，高，宽）形状的输入转换成（批量大小，通道 * 高 * 宽）形状的输入。
• Conv2D自动提特征
• MaxPool2D提升平移泛化能力
• Dense + sigmoid来实现非线性拟合

2. AlexNet

net = nn.Sequential()
net.add(nn.Conv2D(channels=6, kernel_size=11, strides=2, activation='relu'),
        nn.MaxPool2D(pool_size=3, strides=2),
        nn.Conv2D(channels=16, kernel_size=3, padding=2, activation='relu'),
        nn.MaxPool2D(pool_size=3, strides=1),
        nn.Conv2D(channels=16, kernel_size=3, padding=1, activation='relu'),
        nn.Conv2D(channels=16, kernel_size=3, padding=1, activation='relu'),
        nn.Conv2D(channels=6, kernel_size=3, padding=1, activation='relu'),
        nn.MaxPool2D(pool_size=3, strides=1),
        nn.Dense(400, activation="relu"),nn.Dropout(0.5),
        nn.Dense(400, activation="relu"),nn.Dropout(0.5),
        nn.Dense(10))

说明

• padding 来使得输入输出高宽一致
• 初始用kernel_size，其实后面研究说明该方法不优
• MaxPool2D strides控制网络宽高大小，可以看到几个conv块保持同一宽高大小提特征，块间用maxpooling逐渐降维
• 更大的FC配合drop out
• activation用relu替换sigmoid, 计算更快，不咋影响输出效果

3. VGG

def vgg_block(num_convs, num_channels):
    blk = nn.Sequential()
    for _ in range(num_convs):