基本信息
- 实验名称：卷积神经网络实验
- 姓名：drailife
- 学号：
- 日期：2022/10/22
文章目录
一、任务1-二维卷积实验
1.1 任务内容
-
任务具体要求
- 手写二维卷积的实现，并在至少一个数据集上进行实验，从训练时间、预测精度、Loss变化等角度分析实验结果（最好使用图表展示）（只用循环几轮即可）
- 使用torch.nn实现二维卷积，并在至少一个数据集上进行实验，从训练时间、预测精度、Loss变化等角度分析实验结果（最好使用图表展示）
- 不同超参数的对比分析（包括卷积层数、卷积核大小、batchsize、lr等）选其中至少1-2个进行分析
-
任务目的
学习手动实现以及torch.nn实现卷积的方法，并完成分析。 -
任务算法或原理介绍
卷积神经网络是一种带有卷积结构的深度神经网络，卷积结构可以减少深层网络占用的内存量，其三个关键的操作，其一是局部感受野，其二是权值共享，其三是 pooling 层，有效的减少了网络的参数个数，缓解了模型的过拟合问题。
卷积神经网络组成
-
任务所用数据集
使用的是提供的车辆分类数据：- 共 1358张车辆图片
- 分别属于汽车、客车和货车三类
- 汽车：779张 客车：218张 货车：360张
- 每类取20%当作测试集
1.2 任务思路及代码
- 构建数据集
- 手动或torch.nn实现二维卷积
- 构建前馈神经网络，损失函数，优化函数
- 使用网络预测结果，得到损失值
- 进行反向传播，和梯度更新
- 对loss、acc等指标进行分析
1.2.0划分数据集
import os.path
import random
import shutil
import numpy as np
"""
按照训练集和测试集 8 : 2的比例划分数据集
划分后的数据集位于原目录下train_data 和 test_data文件夹下
"""
def Split_Train_Test(floder_path):
trainPath = os.path.join(floder_path, 'train_data')
testPath = os.path.join(floder_path, 'test_data')
# 若存在训练集文件夹则先删除
if os.path.exists(os.path.join(floder_path, 'train_data')):
return trainPath, testPath
floders = os.listdir(floder_path) # 获取当前文件夹下所有的文件夹名字
print(f"训练集位置: {trainPath}\n测试集位置: {testPath}")
# 若 训练数据集文件夹不存在则创建
if not os.path.exists(os.path.join(floder_path, 'train_data')) \
or not os.path.exists(os.path.join(floder_path, 'test_data')):
os.mkdir(os.path.join(floder_path, 'train_data'))
os.mkdir(os.path.join(floder_path, 'test_data'))
for floder in floders:
name1 = os.path.join(floder_path, 'train_data') + os.sep + floder
name2 = os.path.join(floder_path, 'test_data') + os.sep + floder
os.mkdir(name1)
os.mkdir(name2)
# 自定义test的数据比例
train_rate = 0.8
for floder in floders:
if floder == 'test_data' or floder == 'train_data':
continue
print(floder)
filesname = os.listdir(os.path.join(floder_path, floder))
filesname = np.array(filesname)
train_picknumber = int(len(filesname) * train_rate)
total_numbers = list(range(len(filesname)))
random.shuffle(total_numbers) # 随机打乱下标
# 获得打乱下标后的训练集和测试集的数据
train_file = filesname[total_numbers[0: train_picknumber]]
test_file = filesname[total_numbers[train_picknumber:]]
# 开始进行拷贝工作
print(f"训练集数量: {train_picknumber}, 测试集数量{len(filesname) - train_picknumber}")
for file in train_file:
if file == "desktop.ini":
continue
path_train = os.path.join(floder_path, floder) + os.sep + file
path_train_copy = os.path.join(floder_path, 'train_data') + os.sep + floder + os.sep + file
shutil.copy(path_train, path_train_copy) # 将文件复制到训练集文件夹
for file in test_file:
if file == "desktop.ini":
continue
path_test = os.path.join(floder_path, floder) + os.sep + file
path_test_copy = os.path.join(floder_path, 'test_data') + os.sep + floder + os.sep + file
shutil.copy(path_test, path_test_copy) # 讲文件复制到测试集文件夹
return trainPath, testPath
import torch
import torchvision.transforms as transforms
from torch.utils.data import DataLoader
from torchvision.datasets import ImageFolder
import matplotlib.pyplot as plt
# 绘制图像的代码
def picture(name, trainl, testl,xlabel='Epoch',ylabel='Loss'):
plt.rcParams["font.sans-serif"]=["SimHei"] #设置字体
plt.rcParams["axes.unicode_minus"]=False #该语句解决图像中的“-”负号的乱码问题
plt.figure(figsize=(7, 3))
color = ['g','r','b','c']
if trainl is not None:
plt.subplot(121)
for i in range(len(name)):
plt.plot(trainl[i], c=color[i],label=name[i])
plt.xlabel(xlabel)
plt.ylabel(ylabel)
plt.title('Train')
plt.legend()
if testl is not None:
plt.subplot(122)
for i in range(len(name)):
plt.plot(testl[i], c=color[i], label=name[i])
plt.xlabel(xlabel)
plt.ylabel(ylabel)
plt.title('Test')
plt.legend()
加载数据集
# 完成加载数据集的操作
device = torch.device("cuda" if torch.cuda.is_available() else "cpu") # 如果有gpu则在gpu上计算 加快计算速度
data_path = "E:\\DataSet\\车辆分类数据集"
train_path,test_path = Split_Train_Test(data_path) # 划分数据集
batch_size = 32
mytransform = transforms.Compose([transforms.Resize((64, 64)), #拉伸到统一大小
transforms.ToTensor(), # 转换为tensor
transforms.Normalize([0.5, 0.5, 0.5], [0.5, 0.5, 0.5])]) # 归一化
traindataset = ImageFolder(root=train_path, transform=mytransform)
traindataloader = DataLoader(dataset=traindataset, shuffle=True, batch_size=batch_size)
testdataset = ImageFolder(root=test_path, transform=mytransform)
testdataloader = DataLoader(dataset=testdataset, shuffle=True, batch_size=batch_size)
Mydata = [traindataset, testdataset, traindataloader, testdataloader]
print(f"train_dataset's length :{len(traindataset)}")
print(f"test_dataset's length :{len(testdataset)}")
训练集位置: E:\DataSet\车辆分类数据集\train_data
测试集位置: E:\DataSet\车辆分类数据集\test_data
bus
训练集数量: 175, 测试集数量44
car
训练集数量: 623, 测试集数量156
truck
训练集数量: 288, 测试集数量72
train_dataset's length :1085
test_dataset's length :272
1.2.1 手动实现二维卷积实验
1. 卷积操作以及多通道输入输出的实现
import torch
from torch.nn import CrossEntropyLoss
import torch.nn as nn
from torch.optim import SGD
import torch.nn.functional as F
# 卷积操作的实现
def conv2d(X, K):
'''
:param X: 样本输入，shape(batch_size,H,W)
:param K: 卷积核，shape(k_h,k_w)
:return: Y 卷积结果，shape(batch_size, H-k_h+1, W-k_w+1)
'''
batch_size, H, W = X.shape
k_h, k_w = K.shape
# 初始化 Y
Y = torch.zeros((batch_size, H - k_h + 1, W - k_w + 1)).to(device)
for i in range(Y.shape[1]):
for j in range(Y.shape[2]):
Y[:, i, j] = (X[:, i: i + k_h, j:j + k_w] * K).sum(dim=2).sum(dim=1)
return Y
# 多通道输入的卷积实现
def conv2d_multi_in(X, K):
'''
:param X: (batch_size, C_in，H，W)代表有C个输入通道
:param K: (C_in, k_h, k_w)
:return: (batch_size, H_out, W_out)
'''
res = conv2d(X[:, 0, :, :], K[0, :, :])
for i in range(1, X.shape[1]): # 多个通道的结果相加
res += conv2d(X[:, i, :, :], K[i, :, :])
return res
# 实现多输出通道
# 输出通道数 = 卷积核个数
def conv2d_multi_in_out(X, K):
'''
:param X: (batch_size, C_in，H，W)代表有C个输入通道
:param K: (K_num, C_in, k_h, k_w) k_num表示卷积核的个数
:return: (batch_size, K_num, H_out, W_out)
'''
return torch.stack([conv2d_multi_in(X, k) for k in K], dim=1)
# 将卷积运算封装成卷积层
class MyConv2D(torch.nn.Module):
def __init__(self, in_channels, out_channels, kernel_size):
super(MyConv2D, self).__init__()
# 初始化卷积层2个参数：卷积核、偏差
if isinstance(kernel_size, int): # 如果kernel size是一个数
kernel_size = (kernel_size, kernel_size)
# weight的shape:(卷积核个数/输出通道数，输入通道数，卷积核高，卷积核宽)
# torch.randn:返回一个符合均值为0，方差为1的正态分布（标准正态分布）中填充随机数的张量
self.weight = torch.nn.Parameter(torch.randn((out_channels, in_channels) + kernel_size))
self.bias = torch.nn.Parameter(torch.randn(out_channels, 1, 1))
def forward(self, x):
return conv2d_multi_in_out(x, self.weight) + self.bias
2. 二维卷积神经网络模型的构建
class MyConvModule(torch.nn.Module):
def __init__(self,num_classes):
super(MyConvModule, self).__init__()
# 定义一层卷积
self.conv = torch.nn.Sequential(
MyConv2D(in_channels=3, out_channels=32, kernel_size=3),
torch.nn.BatchNorm2d(32),
torch.nn.ReLU(inplace=True) # inplace=True表示计算出来的结果会替换掉原来的Tensor
)
# 输出层，将输出通道数变为分类数量
self.fc = torch.nn.Linear(32, num_classes)
def forward(self, X):
# 图片经过一层卷积，输出（batch_size，C_out, H, W)
out = self.conv(X)
# 使用平均池化层将图片大小变为1*1(图片原大小64*64,卷积后为62*62
out = F.avg_pool2d(out, 62)
# 将out从shape batch_size*32*1*1变为batch_size*32
out = out.squeeze()
# 输入到全连接层
out = self.fc(out)
return out
net = MyConvModule(num_classes=3)
print("网络的结构\n", net)
网络的结构
MyConvModule(
(conv): Sequential(
(0): MyConv2D()
(1): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU(inplace=True)
)
(fc): Linear(in_features=32, out_features=3, bias=True)
)
3. 手动实现的卷积进行训练
import time
"""
定义训练函数
model:定义的模型
epochs:训练总轮数 默认为20
criterion:定义的损失函数，默认为CrossEntropyLoss
lr :学习率 默认为0.01
optimizer:定义的优化函数，默认为SGD
label =0 表示使用自己定义的卷积
"""
# 将训练过程定义为一个函数，方便调用
def train_and_test(model,epochs=40,lr=0.01,optimizer=None,label=1,data=Mydata,y=1):
print(f'当前使用的device为{device}')
traindataset, testdataset, traindataloader, testdataloader = data
print(f"train_dataloader's length :{len(traindataloader)}")
print(f"test_dataloader's length :{len(testdataloader)}")
MyModel = model
if y==1:
print(MyModel)
# 默认的优化函数为SGD 可以根据参数来修改优化函数
if optimizer == None:
optimizer = SGD(MyModel.parameters(), lr=lr)
criterion = CrossEntropyLoss() # 损失函数
criterion = criterion.to(device)
train_all_loss = [] # 记录训练集上得loss变化
test_all_loss = [] # 记录测试集上的loss变化
train_ACC, test_ACC = [], []
trainE_loss, trainE_acc, testE_loss, testE_acc= [], [], [], []
max_train_acc, max_test_acc = 0.0, 0.0 # 记录最高的正确率
begintime = time.time()
for epoch in range(epochs):
train_l, train_epoch_count, test_epoch_count = 0, 0, 0
for i, (data, labels) in enumerate(traindataloader):
data, labels = data.to(device), labels.to(device)
pred = MyModel(data)
# print(pred)
# print(labels)
train_each_loss = criterion(pred, labels).to('cpu') # 计算每次的损失值
optimizer.zero_grad() # 梯度清零
train_each_loss.backward() # 反向传播
optimizer.step() # 梯度更新
train_l += train_each_loss.item()
acc_num = (pred.argmax(dim=1)==labels).sum().to('cpu')
train_epoch_count += acc_num
if label == 0:
trainE_loss.append(round(train_each_loss.item(),4))
trainE_acc.append(acc_num/len(labels))
if (label == 0 and i % 3 == 0):
print(f"Train-Epoch-batch:{epoch+1}-{i+1} train_loss:{trainE_loss[-1]}\
train_acc:{trainE_acc[-1]}")
train_ACC.append(train_epoch_count/len(traindataset))
if train_ACC[-1] > max_train_acc:
max_train_acc = train_ACC[-1]
train_all_loss.append(train_l) # 添加损失值到列表中
with torch.no_grad():
test_loss, test_epoch_count= 0, 0
for data, labels in testdataloader:
data, labels = data.to(device), labels.to(device)
pred = MyModel(data)
test_each_loss = criterion(pred,labels).to('cpu')
test_loss += test_each_loss.item()
acc_num = (pred.argmax(dim=1)==labels).sum()
test_epoch_count += acc_num
if label == 0:
testE_loss.append(round(test_each_loss.item(),4))
testE_acc.append(acc_num/len(labels))
test_all_loss.append(test_loss)
test_ACC.append(test_epoch_count.cpu()/len(testdataset))
if test_ACC[-1] > max_test_acc:
max_test_acc = test_ACC[-1]
if epoch == 0 or epochs == 1 or ((epoch + 1) % (int(epochs / 10)) == 0):
print('epoch: %d | train loss:%.5f | test loss:%.5f | train acc:%5f test acc:%.5f:' % (epoch + 1, train_all_loss[-1], test_all_loss[-1],
train_ACC[-1],test_ACC[-1]))
endtime = time.time()
print("训练集上最高准确率:%.4f\n测试集上最高准确率%4f"%(max_train_acc,max_test_acc))
print("%d轮 总用时: %.3fs" % (epochs, endtime - begintime))
if label == 0:
return trainE_loss,testE_loss, trainE_acc, testE_acc
# 返回训练集和测试集上的 损失值 与 准确率
return train_all_loss,test_all_loss,train_ACC,test_ACC
# 开始训练 训练轮数 2
net = MyConvModule(num_classes=3)
net = net.to(device)
trainE_loss, testE_loss, trainE_acc, testE_acc= train_and_test(model=net, epochs= 1, lr = 0.01,label=0)
当前使用的device为cuda
train_dataloader's length :34
test_dataloader's length :9
MyConvModule(
(conv): Sequential(
(0): MyConv2D()
(1): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU(inplace=True)
)
(fc): Linear(in_features=32, out_features=3, bias=True)
)
Train-Epoch-batch:1-1 train_loss:1.1104 train_acc:0.125
Train-Epoch-batch:1-4 train_loss:1.1272 train_acc:0.1875
Train-Epoch-batch:1-7 train_loss:1.1304 train_acc:0.125
Train-Epoch-batch:1-10 train_loss:1.0677 train_acc:0.40625
Train-Epoch-batch:1-13 train_loss:1.0726 train_acc:0.46875
Train-Epoch-batch:1-16 train_loss:1.0457 train_acc:0.40625
Train-Epoch-batch:1-19 train_loss:0.9861 train_acc:0.4375
Train-Epoch-batch:1-22 train_loss:0.9891 train_acc:0.46875
Train-Epoch-batch:1-25 train_loss:1.028 train_acc:0.5
Train-Epoch-batch:1-28 train_loss:1.0002 train_acc:0.59375
Train-Epoch-batch:1-31 train_loss:0.9778 train_acc:0.5625
Train-Epoch-batch:1-34 train_loss:0.988 train_acc:0.5862069129943848
epoch: 1 | train loss:35.24691 | test loss:8.86630 | train acc:0.434101 test acc:0.57353:
训练集上最高准确率:0.4341
测试集上最高准确率0.573529
1轮 总用时: 5368.265s
1.2.2 torch.nn实现二维卷积实验
1. 定义三层卷积结构
import torch.nn as nn
class ConvModule(nn.Module):
def __init__(self,num_classes):
super(ConvModule,self).__init__()
#定义一个三层卷积
self.conv = nn.Sequential(
nn.Conv2d(in_channels=3,out_channels=32,kernel_size=3,stride=1,padding=0),
nn.BatchNorm2d(32),
nn.ReLU(inplace=True),
nn.Conv2d(in_channels=32,out_channels=64,
kernel_size=3,stride=1,padding=0),
nn.BatchNorm2d(64),
nn.ReLU(inplace=True),
nn.Conv2d(in_channels=64,out_channels=128,kernel_size=3,stride=1,padding=0),
nn.BatchNorm2d(128),
nn.ReLU(inplace=True)
)
#输出层，将通道数变为分类数量
self.fc = nn.Linear(128,num_classes)
def forward(self,X):
#图片先经过三层卷积输出维度(batch_size,Cout,H,W)
out= self.conv(X)
#使用平均池化层将图片的大小变为如x1
out = F.avg_pool2d(out,58)
#将张量out从shape batch x 128 X 1 × 1变为 batch x128
out = out.squeeze()
#输入到全连接层将输出的维度变为classes_num
out = self.fc(out)
return out
2. 进行训练
# 开始训练 选取三层卷积层 学习率为 0.01
net11 = ConvModule(num_classes=3)
net11 = net11.to(device)
train_all_loss11,test_all_loss11,\
train_ACC11,test_ACC11 = train_and_test(model=net11, epochs= 40, lr = 0.01)
当前使用的device为cuda
train_dataloader's length :34
test_dataloader's length :9
ConvModule(
(conv): Sequential(
(0): Conv2d(3, 32, kernel_size=(3, 3), stride=(1, 1))
(1): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU(inplace=True)
(3): Conv2d(32, 64, kernel_size=(3, 3), stride=(1, 1))
(4): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): ReLU(inplace=True)
(6): Conv2d(64, 128, kernel_size=(3, 3), stride=(1, 1))
(7): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(8): ReLU(inplace=True)
)
(fc): Linear(in_features=128, out_features=3, bias=True)
)
epoch: 1 | train loss:29.72819 | test loss:6.67427 | train acc:0.674654 test acc:0.69485:
epoch: 4 | train loss:20.59116 | test loss:4.96591 | train acc:0.803687 test acc:0.83088:
epoch: 8 | train loss:17.91992 | test loss:4.43407 | train acc:0.824885 test acc:0.82721:
epoch: 12 | train loss:16.34974 | test loss:3.65870 | train acc:0.840553 test acc:0.84559:
epoch: 16 | train loss:15.75884 | test loss:3.52913 | train acc:0.833180 test acc:0.88603:
epoch: 20 | train loss:14.65891 | test loss:3.47333 | train acc:0.839631 test acc:0.87132:
epoch: 24 | train loss:13.74280 | test loss:4.16301 | train acc:0.849770 test acc:0.78676:
epoch: 28 | train loss:13.40951 | test loss:3.37810 | train acc:0.855300 test acc:0.89338:
epoch: 32 | train loss:12.99643 | test loss:3.19648 | train acc:0.849770 test acc:0.88235:
epoch: 36 | train loss:12.22786 | test loss:2.84197 | train acc:0.871889 test acc:0.88971:
epoch: 40 | train loss:12.69782 | test loss:2.96191 | train acc:0.863594 test acc:0.86397:
训练集上最高准确率:0.8719
测试集上最高准确率0.904412
torch.nn实现前馈网络-多分类任务 40轮 总用时: 221.051s
1.2.3 超参数分析
在本次实验中选取的超参数为batchsize大小以及lr的大小对实验结果的影响
1. 分别设置batchsize大小为 8，16，32， lr =0.01进行实验
其中32已经进行过实验，此次省略，使用的是上面已经存储的变量
# 定义训练函数
def train_dif(batch_size=8, lr=0.01,y=0):
print("超参数为：batch_size: %d lr: %3f"%(batch_size,lr))
traindataset1 = ImageFolder(root=train_path, transform=mytransform)
traindataloader1 = DataLoader(dataset=traindataset1, shuffle=True, batch_size=batch_size)
testdataset1 = ImageFolder(root=test_path, transform=mytransform)
testdataloader1 = DataLoader(dataset=testdataset1, shuffle=True, batch_size=batch_size)
Mydata1 = [traindataset1, testdataset1, traindataloader1, testdataloader1] # 数据
# 开始训练 选取三层卷积层 学习率为 0.01
net3 = ConvModule(num_classes=3)
net3 = net3.to(device)
return train_and_test(model=net3, epochs= 40, lr = 0.01, data=Mydata1, y=y)
# 设置batchsize大小为 8
train_all_loss12,test_all_loss12,\
train_ACC12,test_ACC12 = train_dif(batch_size=8, lr=0.01)
超参数为：batch_size: 8 lr: 0.010000
当前使用的device为cuda
train_dataloader's length :136
test_dataloader's length :34
epoch: 1 | train loss:102.13229 | test loss:22.12454 | train acc:0.686636 test acc:0.71324:
epoch: 4 | train loss:80.14346 | test loss:16.92136 | train acc:0.779724 test acc:0.84559:
epoch: 8 | train loss:69.33475 | test loss:16.29257 | train acc:0.813825 test acc:0.81985:
epoch: 12 | train loss:60.63227 | test loss:15.31760 | train acc:0.835945 test acc:0.83456:
epoch: 16 | train loss:58.91411 | test loss:13.28337 | train acc:0.847926 test acc:0.86397:
epoch: 20 | train loss:51.54635 | test loss:14.85019 | train acc:0.850691 test acc:0.86397:
epoch: 24 | train loss:51.34175 | test loss:11.63106 | train acc:0.853456 test acc:0.90074:
epoch: 28 | train loss:50.93454 | test loss:10.01658 | train acc:0.861751 test acc:0.89706:
epoch: 32 | train loss:48.76670 | test loss:12.80161 | train acc:0.874654 test acc:0.86765:
epoch: 36 | train loss:46.95270 | test loss:12.09425 | train acc:0.877419 test acc:0.86397:
epoch: 40 | train loss:40.98230 | test loss:13.95157 | train acc:0.894009 test acc:0.84559:
训练集上最高准确率:0.8940
测试集上最高准确率0.919118
torch.nn实现前馈网络-多分类任务 40轮 总用时: 210.240s
# 设置batchsize大小为 16
train_all_loss13,test_all_loss13,\
train_ACC13,test_ACC13 =train_dif(batch_size=16, lr=0.01)
超参数为：batch_size: 16 lr: 0.010000
当前使用的device为cuda
train_dataloader's length :68
test_dataloader's length :17
epoch: 1 | train loss:54.79314 | test loss:11.29648 | train acc:0.676498 test acc:0.75368:
epoch: 4 | train loss:38.28343 | test loss:8.65484 | train acc:0.806452 test acc:0.83456:
epoch: 8 | train loss:35.56350 | test loss:8.14638 | train acc:0.805530 test acc:0.82721:
epoch: 12 | train loss:30.63591 | test loss:6.78831 | train acc:0.835945 test acc:0.86765:
epoch: 16 | train loss:28.72353 | test loss:6.63146 | train acc:0.841475 test acc:0.84559:
epoch: 20 | train loss:26.21183 | test loss:6.36954 | train acc:0.863594 test acc:0.88603:
epoch: 24 | train loss:25.70602 | test loss:5.46882 | train acc:0.855300 test acc:0.88235:
epoch: 28 | train loss:22.86622 | test loss:4.98422 | train acc:0.878341 test acc:0.90074:
epoch: 32 | train loss:22.14363 | test loss:5.15720 | train acc:0.888479 test acc:0.90074:
epoch: 36 | train loss:21.66097 | test loss:5.95625 | train acc:0.882949 test acc:0.85294:
epoch: 40 | train loss:21.41390 | test loss:4.29927 | train acc:0.888479 test acc:0.90809:
训练集上最高准确率:0.8959
测试集上最高准确率0.915441
torch.nn实现前馈网络-多分类任务 40轮 总用时: 197.100s
1.3 实验结果分析
1. 手动实现二维卷积实验分析
将训练的结果（测试集准确率，训练集准确率）绘制成图,如下：
plt.figure(figsize=(7,3))
plt.title("Train")
plt.subplot(121)
plt.plot([i for i in range(34) if i %3 == 0],trainE_loss)
plt.xlabel('batch')
plt.ylabel('Loss')
plt.subplot(122)
plt.plot([i for i in range(34) if i %3 == 0],trainE_acc)
plt.xlabel('batch')
plt.ylabel('ACC')
Text(0, 0.5, 'ACC')
由训练结果可知，手动实现的卷积网络在运行40轮的情况下 训练集上最高准确率:0.4341, 测试集上最高准确率0.5735，总用时: 5368.265s
| Task | Epoch | Total Time | Train Max ACC | Test Max ACC |
|---|---|---|---|---|
| 手动构建卷积 | 1 | 5368.265s | 0.4341 | 0.5735 |
一轮的时间很久，即使是放在GPU上运行，也需要5368.265s，说明手动实现的卷积的计算效率低，分析原因：卷积操作使用for循环实现，而不是矩阵操作（img2col算法）。
由上面绘制的训练集上一轮中每个batch的损失值和正确率我们知道：
在总体趋势上，损失值越来越小，对应的正确率在不断攀升。
2.torch.nn实现的二维卷积实验分析
将训练的结果（训练集loss, 测试集loss, 测试集准确率，训练集准确率）绘制成图,如下：
picture(['Loss'], [train_all_loss11],[test_all_loss11])
picture(['ACC'], [train_ACC11],[test_ACC11],ylabel='ACC')
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-a6pevAmM-1692348195529)(output_31_0.png)]
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-A0dhC44b-1692348195530)(output_31_1.png)]
由训练结果可知，torch.nn实现的卷积网络在运行40轮的情况下 训练集上最高准确率:0.8959, 测试集上最高准确率0.915441，总用时: 197.100s
| Task | Epoch | Total Time | Train Max ACC | Test Max ACC |
|---|---|---|---|---|
| nn构建卷积 | 40 | 221.051s | 0.8719 | 0.904412 |
由上图所示，我们可以知道：
训练集和测试集上的loss值逐步下降，有时会出现跳变的情况，但之后又恢复正常
训练集和测试集的准确率逐步上升，但同时回出现正确率下降的情况，之后恢复正常
出现该种情况的原因可能是到达了局部的最优点。
与手动实现的卷积对比：
其中最主要的特点就是，torch.nn构建的卷积运行速度很快，在四十轮的情况下，只需要221.051s，远远小于手动实现的卷积一轮所需的实现5368.265s，效率大约为手动的1000倍。
其次torch.nn实现的卷积效率较高，在一轮之后的训练集和测试集上正确率分别为0.674和0.694，高于手动实现的卷积，0.4341和0.5735。
3.超参数分析分析
将训练的结果（训练集loss, 测试集loss, 测试集准确率，训练集准确率）绘制成图,如下：
设置batch_size分别为 8，16，32, 64 lr = 0.01
name1 = ['batchsize = 8','batchsize = 16','batchsize = 32']
trainloss1 = [train_all_loss12,train_all_loss13,train_all_loss11]
testloss1 = [test_all_loss12,test_all_loss13,test_all_loss11]
picture(name1, trainloss1, testloss1)
trainacc1 = [train_ACC12,train_ACC13,train_ACC11]
testacc1 = [test_ACC12,test_ACC13,test_ACC11]
picture(name1, trainacc1, testacc1,ylabel='ACC')
由训练的结果，我们可以得到如下图表：
| Task | Epoch | Total Time | Train Max ACC | Test Max ACC |
|---|---|---|---|---|
| batchsize = 8 | 40 | 210.240s | 0.8940 | 0.919118 |
| batchsize = 16 | 40 | 197.100s | 0.8959 | 0.915441 |
| batchsize = 32 | 40 | 221.051s | 0.8719 | 0.904412 |
batch size大小的直接影响很明显：影响一个epoch中需要迭代iteration的次数以及完成每个epoch所需的时间。
我们从上图可以看出batch_size的设置在一定程度上影响着训练loss值得大小，batch_size越小，计算得到得Loss值越大。
同时也影响着运行时间，以及loss值和acc曲线得走向，所以我们在实验中需要设置合适得大小，保证最大得准确率。
更大的batch 会导致渐近测试准确度降低，我们可以通过增加学习率来从更大的批量中恢复丢失的测试精度。
大batch 开始不会“让模型陷入困境”在一些糟糕的局部最优附近。模型可以随时切换到更低的批量大小或更高的学习率，以达到更好的测试精度。
二、任务2-空洞卷积实验
2.1 任务内容
-
任务具体要求
- 使用torch.nn实现空洞卷积，要求dilation满足HDC条件（如1,2,5）且要堆叠多层并在至少一个数据集上进行实验，从训练时间、预测精度、Loss变化等角度分析实验结果（最好使用图表展示）
- 将空洞卷积模型的实验结果与卷积模型的结果进行分析比对，训练时间、预测精度、Loss变化等角度分析
- 不同超参数的对比分析（包括卷积层数、卷积核大小、不同dilation的选择，batchsize、lr等）选其中至少1-2个进行分析（选做）
-
任务目的
利用torch.nn实现空洞卷积完成对应的回归、分类等任务，并进行分析 -
任务算法或原理介绍
空洞卷积利用添加空洞扩大感受野，让原本3x3的卷积核，在相同参数量和计算量下拥有5x5（dilated rate =2）或者更大的感受野，从而无需下采样。扩张卷积（dilated convolutions）又名空洞卷积（atrous convolutions），向卷积层引入了一个称为 “扩张率(dilation rate)”的新参数，该参数定义了卷积核处理数据时各值的间距 -
任务所用数据集（若此前已介绍过则可略）
见任务一
2.2 任务思路及代码
- 构建回归、二分类、多分类数据集
- 利用torch.nn构建实现空洞卷积
- 构建前馈神经网络，损失函数，优化函数
- 使用网络预测结果，得到损失值
- 进行反向传播，和梯度更新
- 对loss、acc等指标进行分析
2.2.1 torch.nn实现空洞卷积
#pytorch封装卷积层
class D_ConvModule(nn.Module):
def __init__(self,num_classes):
super(D_ConvModule,self).__init__()
#定义六层卷积层
#两层HDC（1,2,5,1,2,5）
self.conv = nn.Sequential(
#第一层 (3-1)*1+1=3 （64-3)/1 + 1 =62
nn.Conv2d(in_channels = 3,out_channels = 32,kernel_size = 3 , stride = 1,padding=0,dilation=1),
nn.BatchNorm2d(32),
nn.ReLU(inplace=True),
#第二层 (3-1)*2+1=5 （62-5)/1 + 1 =58
nn.Conv2d(in_channels = 32,out_channels = 32,kernel_size = 3 , stride = 1,padding=0,dilation=2),
nn.BatchNorm2d(32),
nn.ReLU(inplace=True),
#第三层 (3-1)*5+1=11 (58-11)/1 +1=48
nn.Conv2d(in_channels = 32,out_channels = 64,kernel_size = 3 , stride = 1,padding=0,dilation=5),
nn.BatchNorm2d(64),
nn.ReLU(inplace=True),
#第四层(3-1)*1+1=3 （48-3)/1 + 1 =46
nn.Conv2d(in_channels = 64,out_channels = 64,kernel_size = 3 , stride = 1,padding=0,dilation=1),
nn.BatchNorm2d(64),
nn.ReLU(inplace=True),
#第五层 (3-1)*2+1=5 （46-5)/1 + 1 =42
nn.Conv2d(in_channels = 64,out_channels = 64,kernel_size = 3 , stride = 1,padding=0,dilation=2),
nn.BatchNorm2d(64),
nn.ReLU(inplace=True),
#第六层 (3-1)*5+1=11 (42-11)/1 +1=32
nn.Conv2d(in_channels = 64,out_channels = 128,kernel_size = 3 , stride = 1,padding=0,dilation=5),
nn.BatchNorm2d(128),
nn.ReLU(inplace=True)
)
#输出层,将通道数变为分类数量
self.fc = nn.Linear(128,num_classes)
def forward(self,x):
#图片经过三层卷积，输出维度变为(batch_size,C_out,H,W)
out = self.conv(x)
#使用平均池化层将图片的大小变为1x1,第二个参数为最后输出的长和宽（这里默认相等了）
out = F.avg_pool2d(out,32)
#将张量out从shape batchx128x1x1 变为 batch x128
out = out.squeeze()
#输入到全连接层将输出的维度变为3
out = self.fc(out)
return out
定义训练函数
# 开始训练 选取三层卷积层 学习率为 0.01
net21 = D_ConvModule(num_classes=3)
net21 = net21.to(device)
train_all_loss21,test_all_loss21,\
train_ACC21,test_ACC21=train_and_test(model=net21, epochs= 40, lr = 0.01)
当前使用的device为cuda
train_dataloader's length :34
test_dataloader's length :9
D_ConvModule(
(conv): Sequential(
(0): Conv2d(3, 32, kernel_size=(3, 3), stride=(1, 1))
(1): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU(inplace=True)
(3): Conv2d(32, 32, kernel_size=(3, 3), stride=(1, 1), dilation=(2, 2))
(4): BatchNorm2d(32, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(5): ReLU(inplace=True)
(6): Conv2d(32, 64, kernel_size=(3, 3), stride=(1, 1), dilation=(5, 5))
(7): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(8): ReLU(inplace=True)
(9): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1))
(10): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(11): ReLU(inplace=True)
(12): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), dilation=(2, 2))
(13): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(14): ReLU(inplace=True)
(15): Conv2d(64, 128, kernel_size=(3, 3), stride=(1, 1), dilation=(5, 5))
(16): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(17): ReLU(inplace=True)
)
(fc): Linear(in_features=128, out_features=3, bias=True)
)
epoch: 1 | train loss:26.70525 | test loss:5.36878 | train acc:0.721659 test acc:0.79779:
epoch: 4 | train loss:13.51521 | test loss:3.15287 | train acc:0.845161 test acc:0.88235:
epoch: 8 | train loss:9.49754 | test loss:2.28351 | train acc:0.899539 test acc:0.91544:
epoch: 12 | train loss:7.36988 | test loss:1.97999 | train acc:0.937327 test acc:0.93750:
epoch: 16 | train loss:6.87325 | test loss:1.88865 | train acc:0.939170 test acc:0.91544:
epoch: 20 | train loss:5.49881 | test loss:1.28231 | train acc:0.950230 test acc:0.95588:
epoch: 24 | train loss:4.56003 | test loss:1.58155 | train acc:0.957604 test acc:0.95956:
epoch: 28 | train loss:4.42816 | test loss:1.47434 | train acc:0.960369 test acc:0.94118:
epoch: 32 | train loss:3.21830 | test loss:1.02400 | train acc:0.973272 test acc:0.96324:
epoch: 36 | train loss:2.60041 | test loss:1.20375 | train acc:0.974194 test acc:0.95588:
epoch: 40 | train loss:2.83071 | test loss:1.23331 | train acc:0.974194 test acc:0.95221:
训练集上最高准确率:0.9742
测试集上最高准确率0.963235
torch.nn实现前馈网络-多分类任务 40轮 总用时: 689.417s
2.2.2 不同超参数对比
选取 lr 进行对比
1. 设置lr = 0.1，batch_size = 32
net22 = D_ConvModule(num_classes=3)
net22 = net22.to(device)
train_all_loss22,test_all_loss22,\
train_ACC22,test_ACC22=train_and_test(model=net22, epochs= 40, lr = 0.1,y=0)
当前使用的device为cuda
train_dataloader's length :34
test_dataloader's length :9
epoch: 1 | train loss:19.26387 | test loss:4.08066 | train acc:0.767742 test acc:0.81985:
epoch: 4 | train loss:9.42635 | test loss:2.29155 | train acc:0.894009 test acc:0.88971:
epoch: 8 | train loss:5.60199 | test loss:1.55006 | train acc:0.947465 test acc:0.93750:
epoch: 12 | train loss:5.12292 | test loss:1.46143 | train acc:0.949309 test acc:0.93750:
epoch: 16 | train loss:3.85429 | test loss:1.80634 | train acc:0.953917 test acc:0.90074:
epoch: 20 | train loss:3.00328 | test loss:1.21638 | train acc:0.972350 test acc:0.94853:
epoch: 24 | train loss:1.81122 | test loss:0.96657 | train acc:0.981567 test acc:0.97059:
epoch: 28 | train loss:2.57359 | test loss:1.43241 | train acc:0.970507 test acc:0.94485:
epoch: 32 | train loss:1.33876 | test loss:0.84953 | train acc:0.987097 test acc:0.96691:
epoch: 36 | train loss:0.98327 | test loss:0.75034 | train acc:0.988018 test acc:0.97059:
epoch: 40 | train loss:1.14858 | test loss:0.95393 | train acc:0.989862 test acc:0.95956:
训练集上最高准确率:0.9908
测试集上最高准确率0.981618
torch.nn实现前馈网络-多分类任务 40轮 总用时: 726.899s
2. 设置 lr=0.7, batch_size=32
net23 = D_ConvModule(num_classes=3)
net23 = net23.to(device)
train_all_loss23,test_all_loss23,\
train_ACC23,test_ACC23=train_and_test(model=net23, epochs= 40, lr = 0.7,y=0)
当前使用的device为cuda
train_dataloader's length :34
test_dataloader's length :9
epoch: 1 | train loss:33.26175 | test loss:5.00954 | train acc:0.670046 test acc:0.79779:
epoch: 4 | train loss:13.64877 | test loss:3.03121 | train acc:0.824885 test acc:0.87868:
epoch: 8 | train loss:8.88657 | test loss:2.25122 | train acc:0.887558 test acc:0.90441:
epoch: 12 | train loss:6.59880 | test loss:1.92961 | train acc:0.925346 test acc:0.91912:
epoch: 16 | train loss:5.86688 | test loss:1.77138 | train acc:0.927189 test acc:0.93015:
epoch: 20 | train loss:3.82153 | test loss:1.99996 | train acc:0.959447 test acc:0.93750:
epoch: 24 | train loss:4.59842 | test loss:1.96706 | train acc:0.948387 test acc:0.94485:
epoch: 28 | train loss:3.79250 | test loss:1.77480 | train acc:0.956682 test acc:0.93750:
epoch: 32 | train loss:2.60004 | test loss:1.48557 | train acc:0.977880 test acc:0.95221:
epoch: 36 | train loss:1.84863 | test loss:1.61103 | train acc:0.984332 test acc:0.95588:
epoch: 40 | train loss:1.83208 | test loss:1.75209 | train acc:0.982488 test acc:0.94853:
训练集上最高准确率:0.9843
测试集上最高准确率0.955882
torch.nn实现前馈网络-多分类任务 40轮 总用时: 721.748s
2.3 实验结果分析
1.空洞卷积实验分析
将训练的结果（训练集loss, 测试集loss, 测试集准确率，训练集准确率）绘制成图,如下：
picture(['Loss'], [train_all_loss21],[test_all_loss21])
picture(['ACC'], [train_ACC21],[test_ACC21])
由训练得结果我们可以得到图表：
| Task | Epoch | Total Time | Train Max ACC | Test Max ACC |
|---|---|---|---|---|
| 空洞卷积 | 40 | 689.417s | 0.9742 | 0.963235 |
由绘制得loss、acc值得趋势图我们可以看出：
训练集和测试集上的loss值逐步下降，准确率随着epoch得增大而逐步上升，但同时回出现正确率下降的情况，应该是在最优解附近徘徊，在测试集上得最高准确率可以到达0.963
由于用空洞卷积得层数较多为6层，增加了计算量，所以运行时间较长为689.417s
与普通卷积对比
空洞卷积的感受野是指数级别增长的，同样条件下，空洞卷积得感受也大于普通卷积。
空洞卷积由于增加空洞，所以也增加了计算量，故运行时间大于普通卷积（689.417s > 221.051s）,此处也受到模型层数得影响。
在本实验中使用空洞卷积得效果要要好于普通卷积，如结果所示，在测试集上空洞卷积的最大正确率：0.963235，而普通卷积的为0.904412
2.超参数对比分析
将训练的结果（训练集loss, 测试集loss, 测试集准确率，训练集准确率）绘制成图,如下：
name3 = ['lr = 0.01','lr = 0.1','lr = 0.7']
trainloss3 = [train_all_loss21,train_all_loss22,train_all_loss23]
testloss3 = [test_all_loss21,test_all_loss22,test_all_loss23]
picture(name3, trainloss3, testloss3)
trainacc3 = [train_ACC21,train_ACC22,train_ACC23]
testacc3 = [test_ACC21,test_ACC22,test_ACC23]
picture(name3, trainacc3, testacc3)
由训练得结果我们可以得到图表：
| Task | Epoch | Total Time | Train Max ACC | Test Max ACC |
|---|---|---|---|---|
| lr = 0.01 | 40 | 689.417s | 0.9742 | 0.963235 |
| lr = 0.1 | 40 | 197.100s | 0.9908 | 0.981618 |
| lr = 0.7 | 40 | 221.051s | 0.8719 | 0.904412 |
学习率 (learning rate),作为监督学习以及深度学习中重要的超参，它控制网络模型的学习进度,决定这网络能否成功或者需要多久成功找到全局最小值，从而得到全局最优解。
我们知道学习率设置大则导致模型不收敛, 会造成网络不能收敛，在最优值附近徘徊。
如果学习率设置太小，网络收敛非常缓慢，会增大找到最优值的时间。
虽然设置非常小的学习率是可以到达最优解，但是这很可能会进入局部极值点就收敛，因为他的步幅太小。
由图中我们可以看出设置大学习率得模型达到最优解得速度比较小的学习率快。
三、任务3-残差网络实验
3.1 任务内容
-
任务具体要求
-
任务目的
学习构建残差网络 -
任务算法或原理介绍
残差网络是为了解决模型层数增加时出现梯度消失或梯度爆炸的问题而出现的。传统的神经网络中，尤其是图像处理方面，往往使用非常多的卷积层、池化层等，每一层都是从前一层提取特征，所以随着层数增加一般会出现退化等问题。残差网络采取跳跃连接的方法避免了深层神经网络带来的一系列问题。 -
任务所用数据集（若此前已介绍过则可略）
同任务一
3.2 任务思路及代码
- 构建数据集
- 利用torch.nn构建残差网络网络，损失函数，优化函数
- 使用网络预测结果，得到损失值
- 进行反向传播，和梯度更新
- 对loss、acc等指标进行分析
3.2.1 实现残差块
class BasicBlock(nn.Module):
def __init__(self,in_channels,out_channels,stride=[1,1],padding=1):
super(BasicBlock, self).__init__()
# 残差部分
self.layer = nn.Sequential(
nn.Conv2d(in_channels,out_channels,kernel_size=3,stride=stride[0],padding=padding,bias=False),
nn.BatchNorm2d(out_channels),
nn.ReLU(inplace=True), # 原地替换 节省内存开销
nn.Conv2d(out_channels,out_channels,kernel_size=3,stride=stride[1],padding=padding,bias=False),
nn.BatchNorm2d(out_channels)
)
# shortcut 部分
# 由于存在维度不一致的情况 所以分情况
self.shortcut = nn.Sequential()
if stride[0] != 1 or in_channels != out_channels:
self.shortcut = nn.Sequential(
# 卷积核为1 进行升降维
# 注意跳变时 都是stride==2的时候 也就是每次输出信道升维的时候
nn.Conv2d(in_channels, out_channels, kernel_size=1, stride=stride[0], bias=False),
nn.BatchNorm2d(out_channels)
)
def forward(self, x):
out = self.layer(x)
out += self.shortcut(x)
out = F.relu(out)
return out
3.2.2构建残差网络
# 采用bn的网络中，卷积层的输出并不加偏置
class MyResNet(nn.Module):
def __init__(self, num_classes=3) -> None:
super(MyResNet, self).__init__()
self.in_channels = 64
# 第一层作为单独的 因为没有残差快
self.conv1 = nn.Sequential(
nn.Conv2d(3,64,kernel_size=3,stride=1,padding=1,bias=False),
nn.BatchNorm2d(64)
)
#按照图像构造对应残差连接
self.conv2 = self._make_layer(64,[[1,1],[1,1]])
self.conv3 = self._make_layer(128,[[2,1],[1,1]])
self.conv4 = self._make_layer(256,[[2,1],[1,1]])
self.conv5 = self._make_layer(512,[[2,1],[1,1]])
self.avgpool = nn.AdaptiveAvgPool2d((1, 1))
# 该处使用三个全连接层
self.fc = nn.Sequential(
nn.Linear(512, 256),
nn.Linear(256, 128),
nn.Linear(128, num_classes)
)
#构建重复的残差块
def _make_layer(self,out_channels, strides):
layers = []
for stride in strides:
layers.append(BasicBlock(self.in_channels, out_channels, stride))
self.in_channels = out_channels
return nn.Sequential(*layers)
# 图片的大小为 64*64
def forward(self, x):
out = self.conv1(x) # [32, 64, 64, 64]
out = self.conv2(out) # [32, 64, 64, 64]
out = self.conv3(out) # [32, 128, 32, 32]
out = self.conv4(out) # [32, 256, 16, 16]
out = self.conv5(out) # [32, 512, 8, 8]
out = F.avg_pool2d(out,8) # [32, 512, 1, 1]
out = out.squeeze() # [32, 512]
out = self.fc(out)
return out
net31 = MyResNet(num_classes=3)
net31 = net31.to(device)
train_all_loss31,test_all_loss31,\
train_ACC31,test_ACC31=train_and_test(model=net31, epochs= 50, lr = 0.01)
当前使用的device为cuda
train_dataloader's length :34
test_dataloader's length :9
MyResNet(
(conv1): Sequential(
(0): Conv2d(3, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(conv2): Sequential(
(0): BasicBlock(
(layer): Sequential(
(0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU(inplace=True)
(3): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(4): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(shortcut): Sequential()
)
(1): BasicBlock(
(layer): Sequential(
(0): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(1): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU(inplace=True)
(3): Conv2d(64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(4): BatchNorm2d(64, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(shortcut): Sequential()
)
)
(conv3): Sequential(
(0): BasicBlock(
(layer): Sequential(
(0): Conv2d(64, 128, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU(inplace=True)
(3): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(4): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(shortcut): Sequential(
(0): Conv2d(64, 128, kernel_size=(1, 1), stride=(2, 2), bias=False)
(1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(1): BasicBlock(
(layer): Sequential(
(0): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(1): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU(inplace=True)
(3): Conv2d(128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(4): BatchNorm2d(128, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(shortcut): Sequential()
)
)
(conv4): Sequential(
(0): BasicBlock(
(layer): Sequential(
(0): Conv2d(128, 256, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU(inplace=True)
(3): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(4): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(shortcut): Sequential(
(0): Conv2d(128, 256, kernel_size=(1, 1), stride=(2, 2), bias=False)
(1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(1): BasicBlock(
(layer): Sequential(
(0): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(1): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU(inplace=True)
(3): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(4): BatchNorm2d(256, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(shortcut): Sequential()
)
)
(conv5): Sequential(
(0): BasicBlock(
(layer): Sequential(
(0): Conv2d(256, 512, kernel_size=(3, 3), stride=(2, 2), padding=(1, 1), bias=False)
(1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU(inplace=True)
(3): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(4): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(shortcut): Sequential(
(0): Conv2d(256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False)
(1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
)
(1): BasicBlock(
(layer): Sequential(
(0): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(1): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
(2): ReLU(inplace=True)
(3): Conv2d(512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False)
(4): BatchNorm2d(512, eps=1e-05, momentum=0.1, affine=True, track_running_stats=True)
)
(shortcut): Sequential()
)
)
(avgpool): AdaptiveAvgPool2d(output_size=(1, 1))
(fc): Sequential(
(0): Linear(in_features=512, out_features=256, bias=True)
(1): Linear(in_features=256, out_features=128, bias=True)
(2): Linear(in_features=128, out_features=3, bias=True)
)
)
epoch: 1 | train loss:24.99614 | test loss:4.08676 | train acc:0.706912 test acc:0.84191:
epoch: 5 | train loss:9.55118 | test loss:1.95206 | train acc:0.891244 test acc:0.93015:
epoch: 10 | train loss:5.64804 | test loss:1.56333 | train acc:0.948387 test acc:0.95588:
epoch: 15 | train loss:2.23537 | test loss:1.11409 | train acc:0.980645 test acc:0.95956:
epoch: 20 | train loss:1.04646 | test loss:1.07088 | train acc:0.990783 test acc:0.95588:
epoch: 25 | train loss:0.73309 | test loss:1.19770 | train acc:0.994470 test acc:0.94485:
epoch: 30 | train loss:0.84809 | test loss:1.09117 | train acc:0.992627 test acc:0.96324:
epoch: 35 | train loss:0.47239 | test loss:1.02973 | train acc:0.994470 test acc:0.95588:
epoch: 40 | train loss:0.58418 | test loss:1.10057 | train acc:0.991705 test acc:0.93750:
epoch: 45 | train loss:0.39100 | test loss:1.24184 | train acc:0.995392 test acc:0.96324:
epoch: 50 | train loss:0.59156 | test loss:1.52541 | train acc:0.992627 test acc:0.94485:
训练集上最高准确率:0.9954
测试集上最高准确率0.963235
torch.nn实现前馈网络-多分类任务 50轮 总用时: 696.463s
3.3实验结果分析
将训练的结果（训练集loss, 测试集loss, 测试集准确率，训练集准确率）绘制成图,如下：
picture(['Loss'], [train_all_loss31],[test_all_loss31])
picture(['ACC'], [train_ACC31],[test_ACC31])
本次设计了17层的残差网络，由程序运行的结果我们得到：
| Task | Epoch | Total Time | Train Max ACC | Test Max ACC |
|---|---|---|---|---|
| 残差网络 | 50 | 696.463s | 0.9954 | 0.963235 |
网络在车辆分类问题上有着很高的准确性，在第二十轮就可以达到 训练集 0.9907的正确率， 测试集 0.95588的正确率。
由上图我们看出，模型很快收敛。 随着训练轮数的增加，训练集和测试集上的loss值在前期不断地减小，正确率在前期不断的上升。
在后期阶段，loss值出现上下浮动的情况，争取率也在最优解的范围徘徊，说明模型已经差不多训练完成。
本次设置了50轮训练，总耗时696.463s 在训练集上的最好正确率为 0.9954 在测试集上的最高正确率为 0.963235
与普通网络做对比
普通网路由于过拟合和梯度消失或者梯度爆炸的原因，会导致深层网络的层数越深，准确度反而降低。
而残差网络具有以下优点：
1、加深网络层数
2、解决梯度消失问题
3、提升模型精度
A1 实验心得
- 本次实验中我学会分别手动构建和利用pytorch构建卷积，空洞卷积，并构建相应的卷积神经网络完成分类任务。
- 学会搭建残差网络，利用残差网络完成车辆分类任务。
- 对比了手动卷积和pytorch构建的卷积的区别，发现pytorch构建的卷积计算小效率更加高效。
- 对比了普通卷积和空洞卷积，空洞卷积相同情况下，具有更大的感受野。
- 对实现的卷积网络和空洞卷积网络进行了超参数对比实验，探究了batch_size和lr对网络结果的影响。
- 学习到很多新的函数以及网络搭建的思想
A2 参考文献
参考课程PPT
1033
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
