现代卷积网络实战系列6：PyTorch从零构建ResNet训练MNIST数据集

机器学习杨卓越

已于 2023-09-26 16:13:17 修改

阅读量538

点赞数 1

分类专栏：现代卷积网络实战文章标签：图像处理现代卷积神经网络 pytorch 人工智能深度学习计算机视觉

于 2023-09-22 19:37:55 首次发布

本文链接：https://blog.csdn.net/weixin_50592077/article/details/133183292

版权

现代卷积网络实战专栏收录该内容

7 篇文章 1 订阅

订阅专栏

🌈🌈🌈现代卷积网络实战系列总目录

本篇文章的代码运行界面均在Pycharm中进行
本篇文章配套的代码资源已经上传

1、MNIST数据集处理、加载、网络初始化、测试函数
2、训练函数、PyTorch构建LeNet网络
3、PyTorch从零构建AlexNet训练MNIST数据集
4、PyTorch从零构建VGGNet训练MNIST数据集
5、PyTorch从零构建GoogLeNet训练MNIST数据集
6、PyTorch从零构建ResNet训练MNIST数据集

16、ResNet

2015 年何恺明等人提出 ResNet模型，在 ImageNet 的图像分类比赛中获得了冠军，其网络深度高达 152 层。从 ResNet 模型与 VGGNet 模型的对比可以看出，更深层次的结构可以提高网络提取特征的能力和分类性能，但这并不意味着网络可以无限加深。当网络深度达到一定程度后，继续加深网络会导致网络性能退化。为了解决深层网络性能退化问题，在网络中引入残差机制构建了残差块结构，使浅层网络可以通过残差边直接传输到深层网络。残差块结构如图所示：

在这里插入图片描述
ResNet 的核心在于残差模块中的恒等映射结构，该结构既包含了主路竖向连接，又包含支路横向连接。当网络层数过多导致主路竖向连接出现梯度消失问题时，支路横向连接可以保证梯度信息的有效性，防止网络层数过深导致的梯度消失和性能退化。

17、网络架构

ResNet(
(conv1): Conv2d(1, 64, kernel_size=(1, 1), stride=(1, 1))
(maxpool1): MaxPool2d(kernel_size=3, stride=2, padding=1, dilation=1, ceil_mode=False)
(resblock1): Residual(
(conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(conv3): Conv2d(64, 64, kernel_size=(1, 1), stride=(1, 1))
(bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=False, track_running_stats=True)
(bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=False, track_running_stats=True)
(relu1): ReLU()
(relu2): ReLU()
)
(resblock2): Residual(
(conv1): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(conv2): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(conv3): Conv2d(64, 64, kernel_size=(1, 1), stride=(1, 1))
(bn1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=False, track_running_stats=True)
(bn2): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=False, track_running_stats=True)
(relu1): ReLU()
(relu2): ReLU()
)
(resblock3): Residual(
(conv1): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1))
(conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(conv3): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2))
(bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=False, track_running_stats=True)
(bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=False, track_running_stats=True)
(relu1): ReLU()
(relu2): ReLU()
)
(resblock4): Residual(
(conv1): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(conv2): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(conv3): Conv2d(128, 128, kernel_size=(1, 1), stride=(1, 1))
(bn1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=False, track_running_stats=True)
(bn2): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=False, track_running_stats=True)
(relu1): ReLU()
(relu2): ReLU()
)
(resblock5): Residual(
(conv1): Conv2d(128, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(conv3): Conv2d(128, 256, kernel_size=(1, 1), stride=(1, 1))
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=False, track_running_stats=True)
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=False, track_running_stats=True)
(relu1): ReLU()
(relu2): ReLU()
)
(resblock6): Residual(
(conv1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(conv2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(conv3): Conv2d(256, 256, kernel_size=(1, 1), stride=(1, 1))
(bn1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=False, track_running_stats=True)
(bn2): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=False, track_running_stats=True)
(relu1): ReLU()
(relu2): ReLU()
)
(resblock7): Residual(
(conv1): Conv2d(256, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(conv3): Conv2d(256, 512, kernel_size=(1, 1), stride=(1, 1))
(bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=False, track_running_stats=True)
(bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=False, track_running_stats=True)
(relu1): ReLU()
(relu2): ReLU()
)
(resblock8): Residual(
(conv1): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(conv2): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
(conv3): Conv2d(512, 512, kernel_size=(1, 1), stride=(1, 1))
(bn1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=False, track_running_stats=True)
(bn2): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=False, track_running_stats=True)
(relu1): ReLU()
(relu2): ReLU()
)
(avgpool): AdaptiveAvgPool2d(output_size=(1, 1))
(fc): Linear(in_features=512, out_features=10, bias=True)
)

18、PyTorch构建残差块Residual

class Residual(nn.Module):
    def __init__(self, in_channel, out_channel, stride, upsamlpe):
        super(Residual, self).__init__()
        self.conv1 = nn.Conv2d(in_channel, out_channel, kernel_size=3, stride=stride, padding=1)
        self.conv2 = nn.Conv2d(out_channel, out_channel, kernel_size=3, stride=1, padding=1)
        self.conv3 = nn.Conv2d(in_channel, out_channel, kernel_size=1, stride=stride)

        self.bn1 = nn.BatchNorm2d(out_channel, affine=False)
        self.bn2 = nn.BatchNorm2d(out_channel, affine=False)

        self.relu1 = nn.ReLU()
        self.relu2 = nn.ReLU()

    def forward(self, x):
        out = self.relu1(self.bn1(self.conv1(x)))

        out = self.bn2(self.conv2(out))

        x = self.conv3(x)
        out = self.relu2(out + x)

        # print(out.shape)

        return out

19、PyTorch构建ResNet

class ResNet(nn.Module):
    def __init__(self, num_classes):
        super(ResNet, self).__init__()
        self.conv1 = nn.Conv2d(1, 64, kernel_size=1)
        self.maxpool1 = nn.MaxPool2d(3, stride=2, padding=1)

        self.resblock1 = Residual(64, 64, 1, True)
        self.resblock2 = Residual(64, 64, 1, True)
        self.resblock3 = Residual(64, 128, 2, True)
        self.resblock4 = Residual(128, 128, 1, True)
        self.resblock5 = Residual(128, 256, 1, True)
        self.resblock6 = Residual(256, 256, 1, True)
        self.resblock7 = Residual(256, 512, 1, True)
        self.resblock8 = Residual(512, 512, 1, True)
        self.avgpool = nn.AdaptiveAvgPool2d((1, 1))
        self.fc = nn.Linear(512, num_classes)

    def forward(self, x):
        x = self.maxpool1(self.conv1(x))
        x = self.resblock1(x)
        x = self.resblock2(x)
        x = self.resblock3(x)
        x = self.resblock4(x)
        x = self.resblock5(x)
        x = self.resblock6(x)
        x = self.resblock7(x)
        x = self.resblock8(x)

        x = self.avgpool(x)
        x = x.reshape(x.shape[0], -1)
        x = self.fc(x)

        return x

D:\conda\envs\pytorch\python.exe A:\0_MNIST\train.py =

Reading data… train_data: (60000, 28, 28) train_label (60000,)
test_data: (10000, 28, 28) test_label (10000,)

Initialize neural network
test loss 2302.5
test accuracy 10.1%

epoch step 1
training time 23.9s
training loss 177.2
test loss 86.3
test accuracy 97.3%

epoch step 2
training time 30.2s
training loss 80.1
test loss 122.0
test accuracy 96.5%

epoch step 3
training time 18.8s
training loss 63.4
test loss 69.1
test accuracy 97.9%

Training finished
3 epoch training time 72.9s
One epoch average training time 24.3s

进程已结束，退出代码为 0