VGG和NIN网络

最新推荐文章于 2021-08-26 14:31:03 发布

顺毛黑起

最新推荐文章于 2021-08-26 14:31:03 发布

阅读量638

点赞数

分类专栏：深度学习（pytorch）文章标签：神经网络深度学习

本文链接：https://blog.csdn.net/Apikaqiu/article/details/104371832

版权

深度学习（pytorch）专栏收录该内容

9 篇文章 3 订阅

订阅专栏

本文主要是学习了Dive-into-DL-PyTorch这本书。因此这篇博客的大部分内容来源于此书。框架使用的是pytorch，开发工具是pycharm
参考动手学深度学习Dive-into-DL-Pytorch
参考链接 https://github.com/ShusenTang/Dive-into-DL-PyTorch
https://github.com/zergtant/pytorch-handbook

VGG网络

VGG块的组成规律是：连续使用数个相同的填充为1、窗口形状为3x3的卷积层后接一个步幅为2、窗口形状为2x2的最大池化层。卷积层保持输入的高和宽不变，池化层对其减半。
VGG网络，有5个卷积块，前两块使用单卷积层，后3块使用双卷积层。第一块的输入通道数是1（因为使用的fashion-mnist数据的通道数为1），输出通道数是64.之后每次对输出通道数翻倍，知道变成512.由于这个网络使用了8个卷积层和3个全连接层，所以经常被称为VGG-11.

import time
import torch
from torch import nn, optim
import sys
 
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
#vgg_block函数实现基础的VGG块（可以指定卷积层的数量和输入输出通道数）
def vgg_block(num_convs, in_channels, out_channels):
    blk = []
    for i in range(num_convs):
        if i == 0:
            blk.append(nn.Conv2d(in_channels, out_channels, kernel_size=3, padding=1))
        else:
            blk.append(nn.Conv2d(out_channels, out_channels, kernel_size=3, padding=1))
        blk.append(nn.ReLU())
    blk.append(nn.MaxPool2d(kernel_size=2, stride=2)) # 这里会使宽高减半
    return nn.Sequential(*blk)

conv_arch = ((1, 1, 64), (1, 64, 128), (2, 128, 256), (2, 256, 512), (2, 512, 512))
# 经过5个vgg_block,宽高会减半5次，变成224/32=7
fc_features = 512 * 7 * 7 # c * w * h
fc_hidden_units = 4096 # 任意
#VGG-11的实现

#转换形状
class FlattenLayer(torch.nn.Module):
    def __init__(self):
        super(FlattenLayer, self).__init__()

    def forward(self, x):  # x shape: (batch, *, *, ...)
        return x.view(x.shape[0], -1)
        
def vgg(conv_arch, fc_features, fc_hidden_units=4096):
    net = nn.Sequential()
    #卷积层部分
    for i, (num_convs, in_channels, out_channels) in enumerate(conv_arch):
        #每经过一个vgg_block都会使高和宽减半
         net.add_module("vgg_block_" + str(i+1), vgg_block(num_convs, in_channels, out_channels))
    #全连接部分
    net.add_module("fc", nn.Sequential(FlattenLayer(),
                                       nn.Linear(fc_features, fc_hidden_units),
                                       nn.ReLU(),
                                       nn.Dropout(0.5),
                                       nn.Linear(fc_hidden_units, fc_hidden_units), nn.ReLU(), nn.Dropout(0.5),
                                       nn.Linear(fc_hidden_units, 10)))
    return net
#构造一个高和宽均为224的单通道数据样本观察每层形状
net = vgg(conv_arch, fc_features, fc_hidden_units)
X = torch.rand(1, 1, 224, 224)
# named_children获取一级子模块及其名字(named_modules会返回所有子模块，包括子模块的子模块)
for name, blk in net.named_children():
    X = blk(X)
    print(name, 'output shape: ', X.shape)

输出：

vgg_block_1 output shape:  torch.Size([1, 64, 112, 112])
vgg_block_2 output shape:  torch.Size([1, 128, 56, 56])
vgg_block_3 output shape:  torch.Size([1, 256, 28, 28])
vgg_block_4 output shape:  torch.Size([1, 512, 14, 14])
vgg_block_5 output shape:  torch.Size([1, 512, 7, 7])
fc output shape:  torch.Size([1, 10])

模型框架

ratio = 8
small_conv_arch = [(1, 1, 64//ratio), (1, 64//ratio, 128//ratio), (2, 128//ratio, 256//ratio),
                   (2, 256//ratio, 512//ratio), (2, 512//ratio, 512//ratio)]
net = vgg(small_conv_arch, fc_features // ratio, fc_hidden_units // ratio)
print(net)

输出模型框架

Sequential(
  (vgg_block_1): Sequential(
    (0): Conv2d(1, 8, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (1): ReLU()
    (2): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
  )
  (vgg_block_2): Sequential(
    (0): Conv2d(8, 16, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (1): ReLU()
    (2): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
  )
  (vgg_block_3): Sequential(
    (0): Conv2d(16, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (1): ReLU()
    (2): Conv2d(32, 32, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (3): ReLU()
    (4): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
  )
  (vgg_block_4): Sequential(
    (0): Conv2d(32, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (1): ReLU()
    (2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (3): ReLU()
    (4): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
  )
  (vgg_block_5): Sequential(
    (0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (1): ReLU()
    (2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (3): ReLU()
    (4): MaxPool2d(kernel_size=2, stride=2, padding=0, dilation=1, ceil_mode=False)
  )
  (fc): Sequential(
    (0): FlattenLayer()
    (1): Linear(in_features=3136, out_features=512, bias=True)
    (2): ReLU()
    (3): Dropout(p=0.5, inplace=False)
    (4): Linear(in_features=512, out_features=512, bias=True)
    (5): ReLU()
    (6): Dropout(p=0.5, inplace=False)
    (7): Linear(in_features=512, out_features=10, bias=True)
  )
)

NIN网络

LeNet、AlEXNet和VGG共同之处：先以由卷积层构成的模块充分抽取空间特征，再以由全连接层构成的模块来输出分类结果。AlEXNet和VGG对LeNet的改进主要在于如何对这两个模块加宽（增加通道数）和加深。而网络NiN,它提出了另一个思路，即出串联多个由卷积层和"全连接层"构成的小网络来构建一个深层网络。

NIN使用卷积窗口形状分别为11x11、5x5和3x3的卷积层，输出通道数分别是96、256、384.除了使用NIN块以外，NIN去掉了AlEXNet最后的3个全连接层，取而代之使用了输出通道数等于标签类别数的NIN块，然后使用全局平均池化层对每个通道中所有元素求平均并直接用于分类。这里的全局平均池化层即窗口形状等于输入空间维形状的平均池化层。

'''
NIN块是NIN中的基础块。由一个卷积层加连个充当全连接层的1x1卷积层串联而成。
其中第一个卷积层的超参数可以自行设置，而第二个和第三个卷积层的超参数一般是固定的。
'''
def nin_block(in_channels, out_channels, kernel_size, stride, padding):
    blk = nn.Sequential(nn.Conv2d(in_channels, out_channels, kernel_size, stride, padding),
                        nn.ReLU(),
                        nn.Conv2d(out_channels, out_channels, kernel_size=1),
                        nn.ReLU(),
                        nn.Conv2d(out_channels, out_channels, kernel_size=1),
                        nn.ReLU())
    return blk

NIN模型

class GlobalAvgPool2d(nn.Module):
    # 全局平均池化层可通过将池化窗口形状设置成输入的高和宽实现
    def __init__(self):
        super(GlobalAvgPool2d, self).__init__()

    def forward(self, x):
        return F.avg_pool2d(x, kernel_size=x.size()[2:])

#转换形状
class FlattenLayer(torch.nn.Module):
    def __init__(self):
        super(FlattenLayer, self).__init__()

    def forward(self, x):  # x shape: (batch, *, *, ...)
        return x.view(x.shape[0], -1)
net = nn.Sequential(
    nin_block(1, 96, kernel_size=11, stride=4, padding=0),
    nn.MaxPool2d(kernel_size=3, stride=2),
    nin_block(96, 256, kernel_size=5, stride=1, padding=2),
    nn.MaxPool2d(kernel_size=3, stride=2),
    nin_block(256, 384, kernel_size=3, stride=1, padding=1),
    nn.MaxPool2d(kernel_size=3, stride=2),
    nn.Dropout(0.5),
    #标签类别数是10
    nin_block(384, 10, kernel_size=3, stride=1, padding=1),
    GlobalAvgPool2d(),
    # 将四维的输出转成二维的输出，其形状为（批量大小，10）
    FlattenLayer())
#构建一个数据样本查看每一层的输出形状
X = torch.rand(1, 1, 224, 224)
for name, blk in net.named_children():
    X = blk(X)
    print(name, 'output shape: ', X.shape)

输出结果

0 output shape:  torch.Size([1, 96, 54, 54])
1 output shape:  torch.Size([1, 96, 26, 26])
2 output shape:  torch.Size([1, 256, 26, 26])
3 output shape:  torch.Size([1, 256, 12, 12])
4 output shape:  torch.Size([1, 384, 12, 12])
5 output shape:  torch.Size([1, 384, 5, 5])
6 output shape:  torch.Size([1, 384, 5, 5])
7 output shape:  torch.Size([1, 10, 5, 5])
8 output shape:  torch.Size([1, 10, 1, 1])
9 output shape:  torch.Size([1, 10])