卷积神经网络中各模块的作用

最新推荐文章于 2024-05-21 23:36:04 发布

han_mj

最新推荐文章于 2024-05-21 23:36:04 发布

阅读量1.4k

点赞数 2

分类专栏： pytorch 机器学习计算机视觉文章标签：深度学习卷积神经网络计算机视觉

本文链接：https://blog.csdn.net/qq_41726230/article/details/114743947

版权

机器学习同时被 3 个专栏收录

4 篇文章 1 订阅

订阅专栏

pytorch

3 篇文章 1 订阅

订阅专栏

计算机视觉

3 篇文章 0 订阅

订阅专栏

实验背景

在学习卷积神经网络的时候，总是被其中的许多模块（层）搞糊涂：到底网上说的这些操作是真的能提高测试的准确率吗？那它又是如何提高的呢？带着这样的疑问，我就开始了实验。
本实验的目的是通过针对各个模块的对照试验，得到能够提高网络性能的模块以及原理。

实验环境

python 3.6
pytorch 1.0.0
jupyter 1.0.0
实验数据集：CIFAR10

实验内容

取torchvision.datasets中的数据集CIFAR10，并设置批数量为100，代码如下：

batch_size = 100
train_dataset = dsets.CIFAR10(root = './data', train = True, transform = transforms.ToTensor(), download = True)
test_dataset = dsets.CIFAR10(root = './data', train = False, transform = transforms.ToTensor())
train_loader = torch.utils.data.DataLoader(dataset = train_dataset, batch_size = batch_size, shuffle = True, drop_last = True)
test_loader = torch.utils.data.DataLoader(dataset = test_dataset, batch_size = batch_size, shuffle = False, drop_last = True)

设置网络，首先设置一个原始网络。其网络结构如下：
卷积层(size=3, padding = 1, stride = 1， in_channels = 3, out_channels = 16) ->
最大池化层(size = 2, stride = 2) ->
卷积层(size = 3, padding = 1, stride = 1, in_channels = 16, out_channels = 32) ->
最大池化层(size = 2, stride = 2) ->
卷积层(size = 3, padding = 1, stride = 1, in_channels = 32, out_channels = 64) ->
最大池化层(size = 2, stride = 2) ->
全连接层(in_feature = 1024, out_feature = 10) ->
激活函数层(ReLU)

其代码为如下：

class Net1(nn.Module):
    def __init__(self):
        super(Net1, self).__init__()
        self.conv1 = nn.Conv2d(3, 16, 3, padding = 1, stride = 1)
        self.conv2 = nn.Conv2d(16, 32, 3, padding = 1, stride = 1)
        self.conv3 = nn.Conv2d(32, 64, 3, padding  = 1, stride = 1)
        self.maxpool = nn.MaxPool2d(2, 2)
        self.fc = nn.Linear(1024, 10)
        self.relu = nn.ReLU()
    
    def forward(self, x):
        out = self.conv1(x)
        out = self.maxpool(out)
        out = self.conv2(out)
        out = self.maxpool(out)
        out = self.conv3(out)
        out = self.maxpool(out)
        out = out.view(out.size(0), -1)
        out = self.fc(out)
        out = self.relu(out)
        return out

接下去设置若干个对照组：全连接对照组、卷积对照组、激活函数对照组、批归一层对照组、dropout对照组。每一个组内增加或减少其中的某些模块，观察准确率变化的情况。下面是所有组的网络结构简洁信息：

下面给出所有组内每一个神经网络的具体信息：

全连接对照组——网络2结构：在网络1的基础上增加一层全连接层和激活函数层
卷积层(size=3, padding = 1, stride = 1， in_channels = 3, out_channels = 16) ->
最大池化层(size = 2, stride = 2) ->
卷积层(size = 3, padding = 1, stride = 1, in_channels = 16, out_channels = 32) ->
最大池化层(size = 2, stride = 2) ->
卷积层(size = 3, padding = 1, stride = 1, in_channels = 32, out_channels = 64) ->
最大池化层(size = 2, stride = 2) ->
全连接层(in_feature = 1024, out_feature = 100) ->
激活函数层(ReLU) ->
全连接层(in_feature = 100, out_feature = 10) ->
激活函数层(ReLU)
网络2代码：

class Net2(nn.Module):
    def __init__(self):
        super(Net2, self).__init__()
        self.conv1 = nn.Conv2d(3, 16, 3, padding = 1, stride = 1)
        self.conv2 = nn.Conv2d(16, 32, 3, padding = 1, stride = 1)
        self.conv3 = nn.Conv2d(32, 64, 3, padding  = 1, stride = 1)
        self.maxpool = nn.MaxPool2d(2, 2)
        self.fc1 = nn.Linear(1024, 100)
        self.fc2 = nn.Linear(100, 10)
        self.relu = nn.ReLU()
        
    def forward(self, x):
        out = self.conv1(x)
        out = self.maxpool(out)
        out = self.conv2(out)
        out = self.maxpool(out)
        out = self.conv3(out)
        out = self.maxpool(out)
        out = out.view(out.size(0), -1)
        out = self.fc1(out)
        out = self.relu(out)
        out = self.fc2(out)
        out = self.relu(out)
        return out

全连接对照组——网络3结构：在网络2的基础上删除一层激活函数层
卷积层(size=3, padding = 1, stride = 1， in_channels = 3, out_channels = 16) ->
最大池化层(size = 2, stride = 2) ->
卷积层(size = 3, padding = 1, stride = 1, in_channels = 16, out_channels = 32) ->
最大池化层(size = 2, stride = 2) ->
卷积层(size = 3, padding = 1, stride = 1, in_channels = 32, out_channels = 64) ->
最大池化层(size = 2, stride = 2) ->
全连接层(in_feature = 1024, out_feature = 100) ->
全连接层(in_feature = 100, out_feature = 10) ->
激活函数层(ReLU)
网络3代码：

class Net3(nn.Module):
    def __init__(self):
        super(Net3, self).__init__()
        self.conv1 = nn.Conv2d(3, 16, 3, padding = 1, stride = 1)
        self.conv2 = nn.Conv2d(16, 32, 3, padding = 1, stride = 1)
        self.conv3 = nn.Conv2d(32, 64, 3, padding  = 1, stride = 1)
        self.maxpool = nn.MaxPool2d(2, 2)
        self.fc1 = nn.Linear(1024, 100)
        self.fc2 = nn.Linear(100, 10)
        self.relu = nn.ReLU()
    
    def forward(self, x):
        out = self.conv1(x)
        out = self.maxpool(out)
        out = self.conv2(out)
        out = self.maxpool(out)
        out = self.conv3(out)
        out = self.maxpool(out)
        out = out.view(out.size(0), -1)
        out = self.fc1(out)
        out = self.fc2(out)
        out = self.relu(out)
        return out

全连接对照组——网络4结构：在网络1的基础上增加两个全连接层和两个激活函数层
卷积层(size=3, padding = 1, stride = 1， in_channels = 3, out_channels = 16) ->
最大池化层(size = 2, stride = 2) ->
卷积层(size = 3, padding = 1, stride = 1, in_channels = 16, out_channels = 32) ->
最大池化层(size = 2, stride = 2) ->
卷积层(size = 3, padding = 1, stride = 1, in_channels = 32, out_channels = 64) ->
最大池化层(size = 2, stride = 2) ->
全连接层(in_feature = 1024, out_feature = 500) ->
激活函数层(ReLU)
全连接层(in_feature = 500, out_feature = 100) ->
激活函数层(ReLU)
全连接层(in_feature = 100, out_feature = 10) ->
激活函数层(ReLU)
网络4代码：

class Net4(nn.Module):
    def __init__(self):
        super(Net4, self).__init__()
        self.conv1 = nn.Conv2d(3, 16, 3, padding = 1, stride = 1)
        self.conv2 = nn.Conv2d(16, 32, 3, padding = 1, stride = 1)
        self.conv3 = nn.Conv2d(32, 64, 3, padding  = 1, stride = 1)
        self.maxpool = nn.MaxPool2d(2, 2)
        self.fc1 = nn.Linear(1024, 500)
        self.fc2 = nn.Linear(500, 100)
        self.fc3 = nn.Linear(100, 10)
        self.relu = nn.ReLU()
        
    def forward(self, x):
        out = self.conv1(x)
        out = self.maxpool(out)
        out = self.conv2(out)
        out = self.maxpool(out)
        out = self.conv3(out)
        out = self.maxpool(out)
        out = out.view(out.size(0), -1)
        out = self.fc1(out)
        out = self.relu(out)
        out = self.fc2(out)
        out = self.relu(out)
        out = self.fc3(out)
        out = self.relu(out)
        return out

全连接对照组——网络5结构：在网络5的基础上删除两个激活函数层
卷积层(size=3, padding = 1, stride = 1， in_channels = 3, out_channels = 16) ->
最大池化层(size = 2, stride = 2) ->
卷积层(size = 3, padding = 1, stride = 1, in_channels = 16, out_channels = 32) ->
最大池化层(size = 2, stride = 2) ->
卷积层(size = 3, padding = 1, stride = 1, in_channels = 32, out_channels = 64) ->
最大池化层(size = 2, stride = 2) ->
全连接层(in_feature = 1024, out_feature = 500) ->
全连接层(in_feature = 500, out_feature = 100) ->
全连接层(in_feature = 100, out_feature = 10) ->
激活函数层(ReLU)
网络5代码：

class Net5(nn.Module):
    def __init__(self):
        super(Net5, self).__init__()
        self.conv1 = nn.Conv2d(3, 16, 3, padding = 1, stride = 1)
        self.conv2 = nn.Conv2d(16, 32, 3, padding = 1, stride = 1)
        self.conv3 = nn.Conv2d(32, 64, 3, padding  = 1, stride = 1)
        self.maxpool = nn.MaxPool2d(2, 2)
        self.fc1 = nn.Linear(1024, 500)
        self.fc2 = nn.Linear(500, 100)
        self.fc3 = nn.Linear(100, 10)
        self.relu = nn.ReLU()
        
    def forward(self, x):
        out = self.conv1(x)
        out = self.maxpool(out)
        out = self.conv2(out)
        out = self.maxpool(out)
        out = self.conv3(out)
        out = self.maxpool(out)
        out = out.view(out.size(0), -1)
        out = self.fc1(out)
        out = self.fc2(out)
        out = self.fc3(out)
        out = self.relu(out)
        return out

卷积对照组——网络6结构：在网络1的基础上删除一个卷积层和两个最大池化层
卷积层(size=3, padding = 1, stride = 1, in_channels = 3, out_channels = 16) ->
最大池化层(size = 2, stride = 2) ->
卷积层(size = 3, padding = 1, stride = 1, in_channels = 16, out_channels = 32) ->
最大池化层(size = 2, stride = 2) ->
全连接层(in_feature = 2048, out_feature = 10) ->
激活函数层(ReLU)
网络6代码：

class Net6(nn.Module):
    def __init__(self):
        super(Net6, self).__init__()
        self.conv1 = nn.Conv2d(3, 16, 3, padding = 1, stride = 1)
        self.conv2 = nn.Conv2d(16, 32, 3, padding = 1, stride = 1)
        self.maxpool = nn.MaxPool2d(2, 2)
        self.fc = nn.Linear(2048, 10)
        self.relu = nn.ReLU()
    
    def forward(self, x):
        out = self.conv1(x)
        out = self.maxpool(out)
        out = self.conv2(out)
        out = self.maxpool(out)
        out = out.view(out.size(0), -1)
        out = self.fc(out)
        out = self.relu(out)
        return out

卷积对照组——网络7结构：在网络1的基础上删除两个卷积层和两个最大池化层
卷积层(size=3, padding = 1, stride = 1, in_channels = 3, out_channels = 16) ->
最大池化层(size = 2, stride = 2) ->
全连接层(in_feature = 4096, out_feature = 10) ->
激活函数层(ReLU)
网络7代码：

class Net7(nn.Module):
    def __init__(self):
        super(Net7, self).__init__()
        self.conv1 = nn.Conv2d(3, 16, 3, padding = 1, stride = 1)
        self.maxpool = nn.MaxPool2d(2, 2)
        self.fc = nn.Linear(4096, 10)
        self.relu = nn.ReLU()
        
    def forward(self, x):
        out = self.conv1(x)
        out = self.maxpool(out)
        out = out.view(out.size(0), -1)
        out = self.fc(out)
        out = self.relu(out)
        return out

卷积对照组——网络8结构：在网络1的基础上增加一个卷积层和一个最大池化层
卷积层(size=3, padding = 1, stride = 1， in_channels = 3, out_channels = 16) ->
最大池化层(size = 2, stride = 2) ->
卷积层(size = 3, padding = 1, stride = 1, in_channels = 16, out_channels = 32) ->
最大池化层(size = 2, stride = 2) ->
卷积层(size = 3, padding = 1, stride = 1, in_channels = 32, out_channels = 64) ->
最大池化层(size = 2, stride = 2) ->
卷积层(size = 3, padding = 1, stride = 1, in_channels = 64, out_channels = 128) ->
最大池化层(size = 2, stride = 2) ->
全连接层(in_feature = 512, out_feature = 10) ->
激活函数层(ReLU)
网络8代码：

class Net8(nn.Module):
    def __init__(self):
        super(Net8, self).__init__()
        self.conv1 = nn.Conv2d(3, 16, 3, padding = 1, stride = 1)
        self.conv2 = nn.Conv2d(16, 32, 3, padding = 1, stride = 1)
        self.conv3 = nn.Conv2d(32, 64, 3, padding  = 1, stride = 1)
        self.conv4 = nn.Conv2d(64, 128, 3, padding = 1, stride = 1)
        self.maxpool = nn.MaxPool2d(2, 2)
        self.fc = nn.Linear(512, 10)
        self.relu = nn.ReLU()
    
    def forward(self, x):
        out = self.conv1(x)
        out = self.maxpool(out)
        out = self.conv2(out)
        out = self.maxpool(out)
        out = self.conv3(out)
        out = self.maxpool(out)
        out = self.conv4(out)
        out = self.maxpool(out)
        out = out.view(out.size(0), -1)
        out = self.fc(out)
        out = self.relu(out)
        return out

卷积对照组——网络9结构：在网络1的基础上增加两个卷积层和两个最大池化层
卷积层(size=3, padding = 1, stride = 1， in_channels = 3, out_channels = 16) ->
最大池化层(size = 2, stride = 2) ->
卷积层(size = 3, padding = 1, stride = 1, in_channels = 16, out_channels = 32) ->
最大池化层(size = 2, stride = 2) ->
卷积层(size = 3, padding = 1, stride = 1, in_channels = 32, out_channels = 64) ->
最大池化层(size = 2, stride = 2) ->
卷积层(size = 3, padding = 1, stride = 1, in_channels = 64, out_channels = 128) ->
最大池化层(size = 2, stride = 2) ->
卷积层(size = 3, padding = 1, stride = 1, in_channels = 128, out_channels = 256) ->
最大池化层(size = 2, stride = 2) ->
全连接层(in_feature = 256, out_feature = 10) ->
激活函数层(ReLU)
网络9代码：

class Net9(nn.Module):
    def __init__(self):
        super(Net9, self).__init__()
        self.conv1 = nn.Conv2d(3, 16, 3, padding = 1, stride = 1)
        self.conv2 = nn.Conv2d(16, 32, 3, padding = 1, stride = 1)
        self.conv3 = nn.Conv2d(32, 64, 3, padding  = 1, stride = 1)
        self.conv4 = nn.Conv2d(64, 128, 3, padding = 1, stride = 1)
        self.conv5 = nn.Conv2d(128, 256, 3, padding  = 1, stride = 1)
        self.maxpool = nn.MaxPool2d(2, 2)
        self.fc = nn.Linear(256, 10)
        self.relu = nn.ReLU()
    
    def forward(self, x):
        out = self.conv1(x)
        out = self.maxpool(out)
        out = self.conv2(out)
        out = self.maxpool(out)
        out = self.conv3(out)
        out = self.maxpool(out)
        out = self.conv4(out)
        out = self.maxpool(out)
        out = self.conv5(out)
        out = self.maxpool(out)
        out = out.view(out.size(0), -1)
        out = self.fc(out)
        out = self.relu(out)
        return out

激活函数对照组——网络10结构：在网络1的基础上增加三个激活函数层，加在最大池化层之后
卷积层(size=3, padding = 1, stride = 1， in_channels = 3, out_channels = 16) ->
最大池化层(size = 2, stride = 2) ->
激活函数层(ReLU) ->
卷积层(size = 3, padding = 1, stride = 1, in_channels = 16, out_channels = 32) ->
最大池化层(size = 2, stride = 2) ->
激活函数层(ReLU) ->
卷积层(size = 3, padding = 1, stride = 1, in_channels = 32, out_channels = 64) ->
最大池化层(size = 2, stride = 2) ->
激活函数层(ReLU) ->
全连接层(in_feature = 512, out_feature = 10) ->
激活函数层(ReLU)
网络10代码：

class Net10(nn.Module):
    def __init__(self):
        super(Net10, self).__init__()
        self.conv1 = nn.Conv2d(3, 16, 3, padding = 1, stride = 1)
        self.conv2 = nn.Conv2d(16, 32, 3, padding = 1, stride = 1)
        self.conv3 = nn.Conv2d(32, 64, 3, padding  = 1, stride = 1)
        self.maxpool = nn.MaxPool2d(2, 2)
        self.fc = nn.Linear(1024, 10)
        self.relu = nn.ReLU()
    
    def forward(self, x):
        out = self.conv1(x)
        out = self.maxpool(out)
        out = self.relu(out)
        out = self.conv2(out)
        out = self.maxpool(out)
        out = self.relu(out)
        out = self.conv3(out)
        out = self.maxpool(out)
        out = self.relu(out)
        out = out.view(out.size(0), -1)
        out = self.fc(out)
        out = self.relu(out)
        return out

激活函数对照组——网络11结构：在网络1的基础上增加三个激活函数层，加在卷积层之后
卷积层(size=3, padding = 1, stride = 1， in_channels = 3, out_channels = 16) ->
激活函数层(ReLU) ->
最大池化层(size = 2, stride = 2) ->
卷积层(size = 3, padding = 1, stride = 1, in_channels = 16, out_channels = 32) ->
激活函数层(ReLU) ->
最大池化层(size = 2, stride = 2) ->
卷积层(size = 3, padding = 1, stride = 1, in_channels = 32, out_channels = 64) ->
激活函数层(ReLU) ->
最大池化层(size = 2, stride = 2) ->
全连接层(in_feature = 512, out_feature = 10) ->
激活函数层(ReLU)
网络11代码：

class Net11(nn.Module):
    def __init__(self):
        super(Net11, self).__init__()
        self.conv1 = nn.Conv2d(3, 16, 3, padding = 1, stride = 1)
        self.conv2 = nn.Conv2d(16, 32, 3, padding = 1, stride = 1)
        self.conv3 = nn.Conv2d(32, 64, 3, padding  = 1, stride = 1)
        self.maxpool = nn.MaxPool2d(2, 2)
        self.fc = nn.Linear(1024, 10)
        self.relu = nn.ReLU()
    
    def forward(self, x):
        out = self.conv1(x)
        out = self.relu(out)
        out = self.maxpool(out)
        out = self.conv2(out)
        out = self.relu(out)
        out = self.maxpool(out)
        out = self.conv3(out)
        out = self.relu(out)
        out = self.maxpool(out)
        out = out.view(out.size(0), -1)
        out = self.fc(out)
        out = self.relu(out)
        return out

class Net11(nn.Module):
    def __init__(self):
        super(Net11, self).__init__()
        self.conv1 = nn.Conv2d(3, 16, 3, padding = 1, stride = 1)
        self.conv2 = nn.Conv2d(16, 32, 3, padding = 1, stride = 1)
        self.conv3 = nn.Conv2d(32, 64, 3, padding  = 1, stride = 1)
        self.maxpool = nn.MaxPool2d(2, 2)
        self.fc = nn.Linear(1024, 10)
        self.relu = nn.ReLU()
    
    def forward(self, x):
        out = self.conv1(x)
        out = self.relu(out)
        out = self.maxpool(out)
        out = self.conv2(out)
        out = self.relu(out)
        out = self.maxpool(out)
        out = self.conv3(out)
        out = self.relu(out)
        out = self.maxpool(out)
        out = out.view(out.size(0), -1)
        out = self.fc(out)
        out = self.relu(out)
        return out

激活函数对照组——网络12结构：在网络1的基础上删除一个激活函数层
卷积层(size=3, padding = 1, stride = 1， in_channels = 3, out_channels = 16) ->
最大池化层(size = 2, stride = 2) ->
卷积层(size = 3, padding = 1, stride = 1, in_channels = 16, out_channels = 32) ->
最大池化层(size = 2, stride = 2) ->
卷积层(size = 3, padding = 1, stride = 1, in_channels = 32, out_channels = 64) ->
最大池化层(size = 2, stride = 2) ->
全连接层(in_feature = 512, out_feature = 10)
网络12代码：

class Net12(nn.Module):
    def __init__(self):
        super(Net12, self).__init__()
        self.conv1 = nn.Conv2d(3, 16, 3, padding = 1, stride = 1)
        self.conv2 = nn.Conv2d(16, 32, 3, padding = 1, stride = 1)
        self.conv3 = nn.Conv2d(32, 64, 3, padding  = 1, stride = 1)
        self.maxpool = nn.MaxPool2d(2, 2)
        self.fc = nn.Linear(1024, 10)
        self.relu = nn.ReLU()
    
    def forward(self, x):
        out = self.conv1(x)
        out = self.maxpool(out)
        out = self.conv2(out)
        out = self.maxpool(out)
        out = self.conv3(out)
        out = self.maxpool(out)
        out = out.view(out.size(0), -1)
        out = self.fc(out)
        return out

批归一对照组——网络13结构：在网络1的基础上增加三个BN层
卷积层(size=3, padding = 1, stride = 1， in_channels = 3, out_channels = 16) ->
批归一化层 ->
最大池化层(size = 2, stride = 2) ->
卷积层(size = 3, padding = 1, stride = 1, in_channels = 16, out_channels = 32) ->
批归一化层 ->
最大池化层(size = 2, stride = 2) ->
卷积层(size = 3, padding = 1, stride = 1, in_channels = 32, out_channels = 64) ->
批归一化层 ->
最大池化层(size = 2, stride = 2) ->
全连接层(in_feature = 512, out_feature = 10) ->
激活函数层(ReLU)
网络13代码：

class Net13(nn.Module):
    def __init__(self):
        super(Net13, self).__init__()
        self.conv1 = nn.Conv2d(3, 16, 3, padding = 1, stride = 1)
        self.conv2 = nn.Conv2d(16, 32, 3, padding = 1, stride = 1)
        self.conv3 = nn.Conv2d(32, 64, 3, padding  = 1, stride = 1)
        self.maxpool = nn.MaxPool2d(2, 2)
        self.bn1 = nn.BatchNorm2d(16)
        self.bn2 = nn.BatchNorm2d(32)
        self.bn3 = nn.BatchNorm2d(64)
        self.fc = nn.Linear(1024, 10)
        self.relu = nn.ReLU()
    
    def forward(self, x):
        out = self.conv1(x)
        out = self.bn1(out)
        out = self.maxpool(out)
        out = self.conv2(out)
        out = self.bn2(out)
        out = self.maxpool(out)
        out = self.conv3(out)
        out = self.bn3(out)
        out = self.maxpool(out)
        out = out.view(out.size(0), -1)
        out = self.fc(out)
        out = self.relu(out)
        return out

批归一对照组——网络14结构：在网络1的基础上增加一个BN层
卷积层(size=3, padding = 1, stride = 1， in_channels = 3, out_channels = 16) ->
最大池化层(size = 2, stride = 2) ->
卷积层(size = 3, padding = 1, stride = 1, in_channels = 16, out_channels = 32) ->
最大池化层(size = 2, stride = 2) ->
卷积层(size = 3, padding = 1, stride = 1, in_channels = 32, out_channels = 64) ->
最大池化层(size = 2, stride = 2) ->
全连接层(in_feature = 512, out_feature = 10) ->
批归一化层 ->
激活函数层(ReLU)
网络14代码：

class Net14(nn.Module):
    def __init__(self):
        super(Net14, self).__init__()
        self.conv1 = nn.Conv2d(3, 16, 3, padding = 1, stride = 1)
        self.conv2 = nn.Conv2d(16, 32, 3, padding = 1, stride = 1)
        self.conv3 = nn.Conv2d(32, 64, 3, padding  = 1, stride = 1)
        self.maxpool = nn.MaxPool2d(2, 2)
        self.fc = nn.Linear(1024, 10)
        self.relu = nn.ReLU()
        self.bn = nn.BatchNorm1d(10)
    
    def forward(self, x):
        out = self.conv1(x)
        out = self.maxpool(out)
        out = self.conv2(out)
        out = self.maxpool(out)
        out = self.conv3(out)
        out = self.maxpool(out)
        out = out.view(out.size(0), -1)
        out = self.fc(out)
        out = self.bn(out)
        out = self.relu(out)
        return out

dropout对照组——网络15结构：在网络1的基础上增加一个dropout层（丢弃率=0.5）
卷积层(size=3, padding = 1, stride = 1， in_channels = 3, out_channels = 16) ->
最大池化层(size = 2, stride = 2) ->
卷积层(size = 3, padding = 1, stride = 1, in_channels = 16, out_channels = 32) ->
最大池化层(size = 2, stride = 2) ->
卷积层(size = 3, padding = 1, stride = 1, in_channels = 32, out_channels = 64) ->
最大池化层(size = 2, stride = 2) ->
全连接层(in_feature = 512, out_feature = 10) ->
dropout层(p=0.5) ->
激活函数层(ReLU)
网络15代码：

class Net15(nn.Module):
    def __init__(self):
        super(Net15, self).__init__()
        self.conv1 = nn.Conv2d(3, 16, 3, padding = 1, stride = 1)
        self.conv2 = nn.Conv2d(16, 32, 3, padding = 1, stride = 1)
        self.conv3 = nn.Conv2d(32, 64, 3, padding  = 1, stride = 1)
        self.maxpool = nn.MaxPool2d(2, 2)
        self.fc = nn.Linear(1024, 10)
        self.relu = nn.ReLU()
        self.dropout = nn.Dropout(p = 0.5)
    
    def forward(self, x):
        out = self.conv1(x)
        out = self.maxpool(out)
        out = self.conv2(out)
        out = self.maxpool(out)
        out = self.conv3(out)
        out = self.maxpool(out)
        out = out.view(out.size(0), -1)
        out = self.fc(out)
        out = self.dropout(out)
        out = self.relu(out)
        return out

dropout对照组——网络16结构：在网络1的基础上增加三个dropout层（丢弃率=0.5）
卷积层(size=3, padding = 1, stride = 1， in_channels = 3, out_channels = 16) ->
dropout层(p=0.5) ->
最大池化层(size = 2, stride = 2) ->
卷积层(size = 3, padding = 1, stride = 1, in_channels = 16, out_channels = 32) ->
dropout层(p=0.5) ->
最大池化层(size = 2, stride = 2) ->
卷积层(size = 3, padding = 1, stride = 1, in_channels = 32, out_channels = 64) ->
dropout层(p=0.5) ->
最大池化层(size = 2, stride = 2) ->
全连接层(in_feature = 512, out_feature = 10) ->
激活函数层(ReLU)
网络16代码：

class Net16(nn.Module):
    def __init__(self):
        super(Net16, self).__init__()
        self.conv1 = nn.Conv2d(3, 16, 3, padding = 1, stride = 1)
        self.conv2 = nn.Conv2d(16, 32, 3, padding = 1, stride = 1)
        self.conv3 = nn.Conv2d(32, 64, 3, padding  = 1, stride = 1)
        self.maxpool = nn.MaxPool2d(2, 2)
        self.fc = nn.Linear(1024, 10)
        self.relu = nn.ReLU()
        self.dropout = nn.Dropout(p = 0.5)
    
    def forward(self, x):
        out = self.conv1(x)
        out = self.dropout(out)
        out = self.maxpool(out)
        out = self.conv2(out)
        out = self.dropout(out)
        out = self.maxpool(out)
        out = self.conv3(out)
        out = self.dropout(out)
        out = self.maxpool(out)
        out = out.view(out.size(0), -1)
        out = self.fc(out)
        out = self.relu(out)
        return out

dropout对照组——网络17结构：在网络1的基础上增加一个dropout2d层（丢弃率=0.5, 维度=2）
卷积层(size=3, padding = 1, stride = 1， in_channels = 3, out_channels = 16) ->
最大池化层(size = 2, stride = 2) ->
卷积层(size = 3, padding = 1, stride = 1, in_channels = 16, out_channels = 32) ->
最大池化层(size = 2, stride = 2) ->
卷积层(size = 3, padding = 1, stride = 1, in_channels = 32, out_channels = 64) ->
最大池化层(size = 2, stride = 2) ->
全连接层(in_feature = 512, out_feature = 10) ->
dropout2d层(p=0.5) ->
激活函数层(ReLU)
网络17代码：

class Net17(nn.Module):
    def __init__(self):
        super(Net17, self).__init__()
        self.conv1 = nn.Conv2d(3, 16, 3, padding = 1, stride = 1)
        self.conv2 = nn.Conv2d(16, 32, 3, padding = 1, stride = 1)
        self.conv3 = nn.Conv2d(32, 64, 3, padding  = 1, stride = 1)
        self.maxpool = nn.MaxPool2d(2, 2)
        self.fc = nn.Linear(1024, 10)
        self.relu = nn.ReLU()
        self.dropout = nn.Dropout2d(p = 0.5)
    
    def forward(self, x):
        out = self.conv1(x)
        out = self.maxpool(out)
        out = self.conv2(out)
        out = self.maxpool(out)
        out = self.conv3(out)
        out = self.maxpool(out)
        out = out.view(out.size(0), -1)
        out = self.fc(out)
        out = self.dropout(out)
        out = self.relu(out)
        return out

dropout对照组——网络18结构：在网络1的基础上增加三个dropout2d层（丢弃率=0.5, 维度=2）
卷积层(size=3, padding = 1, stride = 1， in_channels = 3, out_channels = 16) ->
dropout2d层(p=0.5) ->
最大池化层(size = 2, stride = 2) ->
卷积层(size = 3, padding = 1, stride = 1, in_channels = 16, out_channels = 32) ->
dropout2d层(p=0.5) ->
最大池化层(size = 2, stride = 2) ->
卷积层(size = 3, padding = 1, stride = 1, in_channels = 32, out_channels = 64) ->
dropout2d层(p=0.5) ->
最大池化层(size = 2, stride = 2) ->
全连接层(in_feature = 512, out_feature = 10) ->
激活函数层(ReLU)
网络18代码：

class Net18(nn.Module):
    def __init__(self):
        super(Net18, self).__init__()
        self.conv1 = nn.Conv2d(3, 16, 3, padding = 1, stride = 1)
        self.conv2 = nn.Conv2d(16, 32, 3, padding = 1, stride = 1)
        self.conv3 = nn.Conv2d(32, 64, 3, padding  = 1, stride = 1)
        self.maxpool = nn.MaxPool2d(2, 2)
        self.fc = nn.Linear(1024, 10)
        self.relu = nn.ReLU()
        self.dropout = nn.Dropout2d(p = 0.5)
    
    def forward(self, x):
        out = self.conv1(x)
        out = self.dropout(out)
        out = self.maxpool(out)
        out = self.conv2(out)
        out = self.dropout(out)
        out = self.maxpool(out)
        out = self.conv3(out)
        out = self.dropout(out)
        out = self.maxpool(out)
        out = out.view(out.size(0), -1)
        out = self.fc(out)
        out = self.relu(out)
        return out

dropout对照组——网络19结构：在网络1的基础上增加一个dropout层（丢弃率=0.1）
卷积层(size=3, padding = 1, stride = 1， in_channels = 3, out_channels = 16) ->
最大池化层(size = 2, stride = 2) ->
卷积层(size = 3, padding = 1, stride = 1, in_channels = 16, out_channels = 32) ->
最大池化层(size = 2, stride = 2) ->
卷积层(size = 3, padding = 1, stride = 1, in_channels = 32, out_channels = 64) ->
最大池化层(size = 2, stride = 2) ->
全连接层(in_feature = 512, out_feature = 10) ->
dropout层(p=0.1) ->
激活函数层(ReLU)
网络19代码：

class Net19(nn.Module):
    def __init__(self):
        super(Net19, self).__init__()
        self.conv1 = nn.Conv2d(3, 16, 3, padding = 1, stride = 1)
        self.conv2 = nn.Conv2d(16, 32, 3, padding = 1, stride = 1)
        self.conv3 = nn.Conv2d(32, 64, 3, padding  = 1, stride = 1)
        self.maxpool = nn.MaxPool2d(2, 2)
        self.fc = nn.Linear(1024, 10)
        self.relu = nn.ReLU()
        self.dropout = nn.Dropout(p = 0.1)
    
    def forward(self, x):
        out = self.conv1(x)
        out = self.maxpool(out)
        out = self.conv2(out)
        out = self.maxpool(out)
        out = self.conv3(out)
        out = self.maxpool(out)
        out = out.view(out.size(0), -1)
        out = self.fc(out)
        out = self.dropout(out)
        out = self.relu(out)
        return out

dropout对照组——网络20结构：在网络1的基础上增加三个dropout层（丢弃率=0.1）
卷积层(size=3, padding = 1, stride = 1， in_channels = 3, out_channels = 16) ->
dropout层(p=0.1) ->
最大池化层(size = 2, stride = 2) ->
卷积层(size = 3, padding = 1, stride = 1, in_channels = 16, out_channels = 32) ->
dropout层(p=0.1) ->
最大池化层(size = 2, stride = 2) ->
卷积层(size = 3, padding = 1, stride = 1, in_channels = 32, out_channels = 64) ->
dropout层(p=0.1) ->
最大池化层(size = 2, stride = 2) ->
全连接层(in_feature = 512, out_feature = 10) ->
激活函数层(ReLU)
网络20代码：

class Net20(nn.Module):
    def __init__(self):
        super(Net20, self).__init__()
        self.conv1 = nn.Conv2d(3, 16, 3, padding = 1, stride = 1)
        self.conv2 = nn.Conv2d(16, 32, 3, padding = 1, stride = 1)
        self.conv3 = nn.Conv2d(32, 64, 3, padding  = 1, stride = 1)
        self.maxpool = nn.MaxPool2d(2, 2)
        self.fc = nn.Linear(1024, 10)
        self.relu = nn.ReLU()
        self.dropout = nn.Dropout(p = 0.1)
    
    def forward(self, x):
        out = self.conv1(x)
        out = self.dropout(out)
        out = self.maxpool(out)
        out = self.conv2(out)
        out = self.dropout(out)
        out = self.maxpool(out)
        out = self.conv3(out)
        out = self.dropout(out)
        out = self.maxpool(out)
        out = out.view(out.size(0), -1)
        out = self.fc(out)
        out = self.relu(out)
        return out

dropout对照组——网络21结构：在网络1的基础上增加一个dropout2d层（丢弃率=0.1, 维度=2）
卷积层(size=3, padding = 1, stride = 1， in_channels = 3, out_channels = 16) ->
最大池化层(size = 2, stride = 2) ->
卷积层(size = 3, padding = 1, stride = 1, in_channels = 16, out_channels = 32) ->
最大池化层(size = 2, stride = 2) ->
卷积层(size = 3, padding = 1, stride = 1, in_channels = 32, out_channels = 64) ->
最大池化层(size = 2, stride = 2) ->
全连接层(in_feature = 512, out_feature = 10) ->
dropout2d层(p=0.1) ->
激活函数层(ReLU)
网络21代码：

class Net21(nn.Module):
    def __init__(self):
        super(Net21, self).__init__()
        self.conv1 = nn.Conv2d(3, 16, 3, padding = 1, stride = 1)
        self.conv2 = nn.Conv2d(16, 32, 3, padding = 1, stride = 1)
        self.conv3 = nn.Conv2d(32, 64, 3, padding  = 1, stride = 1)
        self.maxpool = nn.MaxPool2d(2, 2)
        self.fc = nn.Linear(1024, 10)
        self.relu = nn.ReLU()
        self.dropout = nn.Dropout2d(p = 0.1)
    
    def forward(self, x):
        out = self.conv1(x)
        out = self.maxpool(out)
        out = self.conv2(out)
        out = self.maxpool(out)
        out = self.conv3(out)
        out = self.maxpool(out)
        out = out.view(out.size(0), -1)
        out = self.fc(out)
        out = self.dropout(out)
        out = self.relu(out)
        return out

dropout对照组——网络22结构：在网络1的基础上增加三个dropout2d层（丢弃率=0.1, 维度=2）
卷积层(size=3, padding = 1, stride = 1， in_channels = 3, out_channels = 16) ->
dropout2d层(p=0.1) ->
最大池化层(size = 2, stride = 2) ->
卷积层(size = 3, padding = 1, stride = 1, in_channels = 16, out_channels = 32) ->
dropout2d层(p=0.1) ->
最大池化层(size = 2, stride = 2) ->
卷积层(size = 3, padding = 1, stride = 1, in_channels = 32, out_channels = 64) ->
dropout2d层(p=0.1) ->
最大池化层(size = 2, stride = 2) ->
全连接层(in_feature = 512, out_feature = 10) ->
激活函数层(ReLU)
网络22代码：

class Net22(nn.Module):
    def __init__(self):
        super(Net22, self).__init__()
        self.conv1 = nn.Conv2d(3, 16, 3, padding = 1, stride = 1)
        self.conv2 = nn.Conv2d(16, 32, 3, padding = 1, stride = 1)
        self.conv3 = nn.Conv2d(32, 64, 3, padding  = 1, stride = 1)
        self.maxpool = nn.MaxPool2d(2, 2)
        self.fc = nn.Linear(1024, 10)
        self.relu = nn.ReLU()
        self.dropout = nn.Dropout2d(p = 0.1)
    
    def forward(self, x):
        out = self.conv1(x)
        out = self.dropout(out)
        out = self.maxpool(out)
        out = self.conv2(out)
        out = self.dropout(out)
        out = self.maxpool(out)
        out = self.conv3(out)
        out = self.dropout(out)
        out = self.maxpool(out)
        out = out.view(out.size(0), -1)
        out = self.fc(out)
        out = self.relu(out)
        return out

设置损失函数、正则化器、训练、测试过程。对每一个网络进行三训练，并记录准确率和LOSS值，对于准确率，求平均值

net = Net22().cuda() # 在训练的时候修改该行调用的类
criterion = nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(net.parameters(), lr = learning_rate)

for epoch in range(num_epochs):
    for i, (images, labels) in enumerate(train_loader):
        images = Variable(images).cuda()
        labels = Variable(labels).cuda()
        optimizer.zero_grad()
        outputs = net(images)
        loss = criterion(outputs, labels)
        loss.backward()
        optimizer.step()
        if (i + 1) % 100 == 0:
            print("Epoch [%d/%d] image[%d/%d] loss:%.4f"%(epoch + 1, num_epochs, i + 1, len(train_dataset) // batch_size, loss.data))

total = 0
correct = 0
for images, labels in test_loader:
    images = Variable(images).cuda()
    outputs = net(images).cuda()
    _, predicted = torch.max(outputs.data, 1)
    correct += (labels == predicted.cpu()).sum()
    total += labels.size(0)
print("Accurancy of the model on the test images:%d %%"%(100 * correct / total))

实验结果

实验结果如下图。
由1、3、5可得，增加全连接层数对准确率的影响并不大，所以大多数模型只在结尾使用一层全连接层。

由1、12、10、11、2、3、4、5可得，在本实验中（图片大小为32*32），并不需要激活函数，使用激活函数可能会导致模型的非线性程度变高，使模型退化。

由1、6、7、8、9可得，当卷积层数过少时，会导致感受野过小，模型学不到参数；而当卷积层数过多时（如编号9），会导致过拟合（LOSS降低，准确率变低）

由1、13、14可得，三层卷积的网络深度并不深，不会引起梯度爆炸/梯度消失，所以添加归一层后准确率并未提高。

由1、15、19可得，因为网络并无过拟合，所以增加dropout不会提高准确率。由15、19、16、20、17、21、18、22可得，将dropout层的丢失率设置太高会使得模型学习不到参数，并且dropout2d的效果明显高于dropout。所以由22行可得，使用dropout2d的最好方法是，设置较小丢失率并多次使用。

在这里插入图片描述