2024年1月15日学习记录——有关resnet18的简单再实现

BARBERUM

已于 2024-01-31 16:28:09 修改

阅读量1.1k

点赞数 38

文章标签：学习深度学习人工智能

于 2024-01-31 16:25:31 首次发布

本文链接：https://blog.csdn.net/weixin_73822830/article/details/135956343

版权

本文详细介绍了ResNet18神经网络的结构，包括残差块的设计与实现，以及在CIFAR10数据集上的训练过程，展示了使用GPU加速训练的效果。最终，经过10个epoch的训练，模型达到了47%的Top1精度。

摘要由CSDN通过智能技术生成

2024年1月15日学习记录

1.有关resnet18重写并训练的任务

resnet本意为resdual net，就是残差神经网络，利用shortcut的连接方式，将特征层隔层连接，在保留原有特征的同时进行深层卷积。可以有效的解决因神经网络层数的叠加而导致的退化问题。

根据以下的逻辑图实现:

在这里插入图片描述

首先图片作为输入，格式为[3,32,32]

经过一个7*7的卷积核和一个最大池化层后进入残差结构层

第一级残差结构层为两个3*3卷积核，不改变通道数，输入与最后输出进行跳接，数值上进行相加。
经过两个一级残差结构层后进入二级残差结构层
1. 第一级残差结构

class BasicBlock1(nn.Module):
    def __init__(self, in_channel, out_channel):
        super(BasicBlock1, self).__init__()
        self.Basicblock = nn.Sequential(
            nn.Conv2d(in_channel, out_channel, 3, 1, 1),
            nn.Conv2d(64, 64, 3, 1, 1)
        )
        self.relu = nn.ReLU()

    def forward(self, x):
        Ret_x = self.Basicblock(x)
        Ret_x = self.relu(Ret_x)
        return x + Ret_x

第二级残差结构层为两个3*3的卷积核，通道数变为128，同时在第一个二级残差结构中的第一层卷积层的步长为2，第二个卷积层步长为1，第二个二级残差结构中卷积层的卷积核步长皆为1。由于通道数的改变以及尺寸的改变，在第一个二级残差结构的跳接中，特征层要经过1**1，步长为2的卷积层，改变其通道数与尺寸才能进行跳接。第二个二级残差结构则不改变尺寸以及通道数。
1. 第二级残差结构

class BasicBlock2_f(nn.Module):
    def __init__(self, in_channel, out_channel):
        super(BasicBlock2_f, self).__init__()
        self.Basicblock = nn.Sequential(
            nn.Conv2d(in_channel, out_channel, 3, 2, 1),
            nn.Conv2d(128, 128, 3, 1, 1)
        )
        self.relu = nn.ReLU()
        self.con = nn.Conv2d(in_channel, out_channel, 1, 2, 0)

    def forward(self, x):
        Ret_x = self.Basicblock(x)
        Ret_x = self.relu(Ret_x)
        don_x = self.con(x)
        return don_x + Ret_x


class BasicBlock2(nn.Module):
    def __init__(self, in_channel, out_channel):
        super(BasicBlock2, self).__init__()
        self.Basicblock = nn.Sequential(
            nn.Conv2d(in_channel, out_channel, 3, 1, 1),
            nn.Conv2d(128, 128, 3, 1, 1)
        )
        self.relu = nn.ReLU()

    def forward(self, x):
        Ret_x = self.Basicblock(x)
        Ret_x = self.relu(Ret_x)
        return Ret_x

经过两个二级残差结构层厚进入三级残差结构层。
三级及四级残差结构层的结构与二级残差层类似，只不过通道数变为256，在第四级中通道数变为512。
1. 第三级残差结构

class BasicBlock3_f(nn.Module):
    def __init__(self, in_channel, out_channel):
        super(BasicBlock3_f, self).__init__()
        self.Basicblock = nn.Sequential(
            nn.Conv2d(in_channel, out_channel, 3, 2, 1),
            nn.Conv2d(256, 256, 3, 1, 1)
        )
        self.relu = nn.ReLU()
        self.con = nn.Conv2d(128, 256, 1, 2, 0)

    def forward(self, x):
        Ret_x = self.Basicblock(x)
        Ret_x = self.relu(Ret_x)
        don_x = self.con(x)
        return don_x + Ret_x


class BasicBlock3(nn.Module):
    def __init__(self, in_channel, out_channel):
        super(BasicBlock3, self).__init__()
        self.Basicblock = nn.Sequential(
            nn.Conv2d(in_channel, out_channel, 3, 1, 1),
            nn.Conv2d(256, 256, 3, 1, 1)
        )
        self.relu = nn.ReLU()

    def forward(self, x):
        Ret_x = self.Basicblock(x)
        Ret_x = self.relu(Ret_x)
        return Ret_x

第四级残差结构

class BasicBlock4_f(nn.Module):
    def __init__(self, in_channel, out_channel):
        super(BasicBlock4_f, self).__init__()
        self.Basicblock = nn.Sequential(
            nn.Conv2d(in_channel, out_channel, 3, 2, 1),
            nn.Conv2d(512, 512, 3, 1, 1)
        )
        self.relu = nn.ReLU()
        self.con = nn.Conv2d(256, 512, 1, 2, 0)

    def forward(self, x):
        Ret_x = self.Basicblock(x)
        Ret_x = self.relu(Ret_x)
        don_x = self.con(x)
        return don_x + Ret_x


class BasicBlock4(nn.Module):
    def __init__(self, in_channel, out_channel):
        super(BasicBlock4, self).__init__()
        self.Basicblock = nn.Sequential(
            nn.Conv2d(in_channel, out_channel, 3, 1, 1),
            nn.Conv2d(512, 512, 3, 1, 1)
        )
        self.relu = nn.ReLU()

    def forward(self, x):
        Ret_x = self.Basicblock(x)
        Ret_x = self.relu(Ret_x)
        return Ret_x

最后经过一个平均池化层再进行展开线性连接。

Resnet18主结构

class myResnet(nn.Module):
    def __init__(self):
        super(myResnet, self).__init__()
        self.con1 = nn.Conv2d(3, 64, 7, 2, 3)
        self.maxpool1 = nn.MaxPool2d(3, 2, 1)
        self.avgpool = nn.AvgPool2d(1)
        self.b1_f = BasicBlock1(64, 64)
        self.b1 = BasicBlock1(64, 64)
        self.b2_f = BasicBlock2_f(64, 128)
        self.b2 = BasicBlock2(128, 128)
        self.b3_f = BasicBlock3_f(128, 256)
        self.b3 = BasicBlock3(256, 256)
        self.b4_f = BasicBlock4_f(256, 512)
        self.b4 = BasicBlock4(512, 512)
        self.avgpool = nn.AdaptiveAvgPool2d((1, 1))
        self.fl = nn.Flatten()
        self.fc = nn.Linear(512, 10)

    def forward(self, x):
        x = self.con1(x)
        x = self.maxpool1(x)
        x = self.b1_f(x)
        x = self.b1(x)
        x = self.b2_f(x)
        x = self.b2(x)
        x = self.b3_f(x)
        x = self.b3(x)
        x = self.b4_f(x)
        x = self.b4(x)
        x = self.avgpool(x)
        x = self.fl(x)
        x = self.fc(x)
        return x

resnet18是指17个卷积层以及最后一个线性连接层。

训练以及验证代码

数据集导入

使用的数据集为CIFAR10数据集

transform = transforms.Compose([    
    transforms.Resize([224, 224]),    
    transforms.ToTensor(),    
    transforms.Normalize([0.5, 0.5, 0.5], 
                         [0.5, 0.5, 0.5])])
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
dataset_train = torchvision.datasets.CIFAR10("data", True, 
                                             transform=transform, 
                                             download=True)
dataset_val = torchvision.datasets.CIFAR10("data", False, 
                                           transform=transform, 
                                           download=True)
dataloader_train = DataLoader(dataset_train, batch_size=100, shuffle=True)
dataloader_val = DataLoader(dataset_val, batch_size=100, shuffle=True)

训练及其验证代码

#查看有无可用cuda
device = torch.device("cuda" if torch.cuda.is_available() else "cpu")
#初始化，并将神经网络放至gpu
net = myResnet().to(device)
#损失函数
cost = torch.nn.CrossEntropyLoss()
#优化器
optim = torch.optim.SGD(net.parameters(), lr=0.1)
#学习速率调节器
scheduler = StepLR(optim, step_size=3, gamma=0.1)

epochs = 10

for epoch in range(epochs):
    net.train()
    running_loss = 0.0
    running_acc = 0.0

    for data in dataloader_train:
        img, label = data
        img, label = img.to(device), label.to(device)
        output = net(img)
        optim.zero_grad()
        #计算误差
        loss = cost(output, label)
        #反向传播
        loss.backward()
        #梯度裁剪
        torch.nn.utils.clip_grad_norm_(net.parameters(), max_norm=1)
        #随机梯度下降优化
        optim.step()
        #累加损失
        running_loss += loss.item()
        #获取预测
        _, pred = torch.max(output.data, 1)
        #计算正确数量
        running_acc += torch.sum(pred == label.data)

    scheduler.step()

    test_corr = 0.0
    test_loss = 0.0
    net.eval()
    with torch.no_grad():
        for data in dataloader_val:
            img, label = data
            img, label = img.to(device) , label.to(device)
            output = net(img)
            #计算损失
            loss = cost(output, label)
            #损失累加
            test_loss += loss.item()
            #获取预测
            _, pred = torch.max(output.data, 1)
            #计算正确数量
            test_corr += torch.sum(pred == label.data)
    #每个训练周期结束后打印周期数
    print("Epoch %d:" % epoch)
    #打印训练损失值以及训练准确率（准确数量占数据集长度之比）
    print("   Training Loss: %.4f, Accuracy: %.2f%%" % (running_loss, running_acc / len(dataset_train) * 100))
    #打印验证损失值以及验证准确率（验证准确数量占数据集长度之比）
    print("   Testing Loss: %.4f, Accuracy: %.2f%%" % (test_loss, test_corr / len(dataset_val) * 100))