传送门:B站 刘二大人 《PyTorch深度学习实践》完结合集_哔哩哔哩_bilibili
文章目录
1.穷举法计算线性模型
知识点:前向传播、损失函数
问题描述:构建线性模型探究x到y的映射关系
主要思路:构建模型 y = w × x y = w \times x y=w×x, 穷举遍历不同的 w w w值,利用 M S E MSE MSE均方差衡量预测值与真实值的大小,绘图寻找使得损失函数最小的权重。
代码实现:
- 构建数据集(输入+标签)
- 定义前向计算和损失计算函数
- 主函数外循环遍历权重,内循环为每次数据前向传播
- 计算平均损失值,绘制平均损失值随着不同权重变化的曲线
import numpy as np
import matplotlib.pyplot as plt
# 1.数据整理
x_data = [1.0, 2.0, 3.0]
y_data = [2.0, 4.0, 6.0]
# 2.前向传播(算y_pred)
def forward(x):
return x * w
# 3.损失函数计算(计算预测值与真实值差距)
def loss(x,y):
y_pred = forward(x)
return (y_pred - y)2
w_list = []
mse_list = []
# 4.设置不同权重,计算损失值并可视化(最优化问题)
for w in np.arange(0.0, 4.1, 0.1): # 大循环遍历权重
print('w=', )
l_sum = 0
for x_val,y_val in zip(x_data,y_data): # 小循环处理所有数据集
y_pred_val = forward(x_val)
loss_val = loss(x_val,y_val)
l_sum += loss_val
print('\t', x_val, y_val, y_pred_val, loss_val)
print('MSE=', l_sum/3)
w_list.append(w)
mse_list.append(l_sum/3)
# 5.绘制不同w下的数据集平均损失值
plt.plot(w_list,mse_list)
plt.ylabel('Loss')
plt.xlabel('w')
plt.show()
np.arange(a, b, c)生成从a到b的等差序列(公差为c)
zip函数将两个迭代器对应元素进行打包成元组
利用plt绘图记得加show()
2.梯度下降法更新权重
2.1梯度下降法
知识点:梯度下降、训练
问题描述:遍历权重主观性太强,需要制定相应的权重更新法则
主要思路:利用梯度下降法进行权重更新,计算损失沿着链式法则对权重的导数(更新方向),选择合适步长进行更新(更新大小)。引入训练的概念,多次循环对同一数据集进行权重更新。
代码实现:
- 定义整体平均损失值函数、以及计算平均梯度函数。
- 主函数构建训练循环,每一轮计算平均损失值和平均梯度,利用平均梯度以及自定义的学习率对权重进行更新
x_data = [1.0, 2.0, 3.0]
y_data = [2.0, 4.0, 6.0]
w = 1.0 # 初始权重猜测
def forward(x):
# 前馈计算
return x * w
def cost(xs, ys):
# 计算平均损失
cost = 0
for x, y in zip(xs, ys):
y_pre = forward(x)
cost += (y_pre - y) 2
return cost / len(xs)
def loss(x, y):
# 损失值
y_pre = forward(x)
return (y_pre - y) 2
def gradient(xs, ys):
# 计算平均梯度
grad = 0
for x, y in zip(xs, ys):
grad += 2 * x * (x * w - y)
return grad / len(xs)
print('Predict (before training)', 4, forward(4))
# 梯度下降
for epoch in range(100):
cost_val = cost(x_data, y_data)
grad_val = gradient(x_data, y_data)
w -= 0.01 * grad_val # 权重更新(每个epoch更新一次)
print('Epoch', epoch, 'w=', w, 'loss=', cost_val)
print('Predict (after training)', 4, forward(4))
2.2 随机梯度下降法
知识点:SGD
随机梯度下降法,minibatch
思想
问题描述:进行样本级的梯度计算,效果优于计算整体平均梯度进行更新
主要思路:不再计算平均梯度进行更新。(逐样本更新权重会提高时间复杂度,后期引入minibatch
的概念,即将整个数据集分成相应小份,每一小份一小份进行计算)
代码实现:主函数构建两层循环,将权重更新代码放入内层循环中
x_data = [1.0, 2.0, 3.0]
y_data = [2.0, 4.0, 6.0]
w = 1.0 # 初始权重猜测
def forward(x):
return x * w
def loss(x, y):
# 随机梯度下降
y_pred = forward(x)
return (y_pred - y) 2
def gradient(x, y):
return 2 * x * (x * w - y)
print('Predict (before training)', 4, forward(4))
# 梯度下降
for epoch in range(100):
for x, y in zip(x_data, y_data):
grad = gradient(x, y)
w -= 0.01 * grad # 权重更新(每个样本更新一次)
l = loss(x, y)
print('Epoch', epoch, 'w=', w, 'loss=', l)
print('Predict (after training)', 4, forward(4))
3.反向传播
知识点:反向传播、torch
问题描述:上述方法需要自定义梯度计算函数,也就是要手动求梯度
主要思路:利用torch
中自动求梯度的功能,进行反向传播更新梯度
代码实现:
- 定义数据集、初始化权重w(定义为
tensor
张量,并将requires_grad
设置为True
) - 构建两层循环,在内循环逐数据处理时需要进行:
- 损失计算
l = loss(x, y)
- 反向传播
l.backward()
- 权重更新
w.data = w.data - 0.01 * w.grad.data
- 梯度清零
w.grad.data.zero_()
- 损失计算
import torch
# 1.数据集
x_data = [1.0, 2.0, 3.0]
y_data = [2.0, 4.0, 6.0]
w = torch.tensor([1.0])
w.requires_grad = True
def forward(x):
return x * w
def loss(x, y):
y_pred = forward(x)
return (y_pred - y)2
print("predict (before training)", forward(4).item())
for epoch in range(100):
for x, y in zip(x_data, y_data):
l = loss(x, y) # l是一个张量,tensor主要是在建立计算图 forward, compute the loss
l.backward()
print('\tgrad:', x, y, w.grad.item())
w.data = w.data - 0.01 * w.grad.data # 权重更新时,注意grad也是一个tensor
w.grad.data.zero_()
print('process:', epoch, l.item()) # 取出loss使用l.item,不要直接使用l(l是tensor会构建计算图)
print("predict (after training)", 4, forward(4).item())
tensor张量包含data和grad两个属性,且data和grad也是张量,故更新w.data时需使用w.grad.data。
grad初始为None,对需要计算梯度的张量设置require_grad为True后,利用backward后自动计算梯度,l.backward()会把计算图中所有需要梯度(grad)的地方都会求出来,然后把梯度都存在对应的待求的参数中,最终计算图被释放。
在print或者只需要tensor的数值时,记得利用item()转化为标量将数值取出来
4.利用PyTorch是实现线性回归
知识点:模型架构
问题描述:利用PyTorch是实现线性回归
代码实现:基本框架包括:数据集准备、模型定义并实例化、损失函数和优化器、模型训练测试
- 数据集准备
- 需要为
tensor
张量矩阵形式,float32
类型。分为输入和标签,每组要进行对应,注意维度含义
- 需要为
- 模型定义并实例化
- 继承
torch.nn.Module
类,定义__init__(self)
和forward(self,x)
- 在
__init__(self)
中利用super
调用父类构造函数,并初始化模型所需要的层 forward(self,x)
进行前向传播- 实例化模型对象
- 继承
- 损失函数与优化器
- 损失函数:计算损失值,进行反向传播
- 优化器:进行梯度清零与更新,需要指定模型总参数、学习率
- 模型训练测试
- 计算输出值、计算损失值、梯度清零、反向传播、梯度更新
import torch
# 1.数据准备
# 矩阵x,y为3行1列,有3个数据,每组数据只有1个特征
x_data = torch.tensor([[1.0], [2.0], [3.0]])
y_data = torch.tensor([[2.0], [4.0], [6.0]])
# 2.模型设计
"""
our model class should be inherit from nn.Module, which is base class for all neural network modules.
member methods __init__() and forward() have to be implemented
class nn.linear contain two member Tensors: weight and bias
class nn.Linear has implemented the magic method __call__(),which enable the instance of the class can
be called just like a function.Normally the forward() will be called
"""
class LinearModel(torch.nn.Module):
def __init__(self):
super(LinearModel, self).__init__()
self.linear = torch.nn.Linear(1, 1)
# (1,1)是指输入x和输出y的特征维度,这里数据集中的x和y的特征都是1维的
# 该线性层需要学习的参数是w和b 获取w/b的方式分别是~linear.weight/linear.bias
def forward(self,x):
y_pred = self.linear(x)
return y_pred
# 3.实例化对象
model = LinearModel()
# 4.损失函数和优化器
criterion = torch.nn.MSELoss(size_average=False)
optimizer = torch.optim.SGD(model.parameters(), lr=0.01)
# 5.训练循环
for epoch in range(500):
y_pred = model(x_data)
loss = criterion(y_pred, y_data)
print(epoch, loss.item())
optimizer.zero_grad()
loss.backward()
optimizer.step()
print('w=', model.linear.weight.item())
print('b=',model.linear.bias.item())
# 5.模型测试
x_test = torch.tensor([[4.0]])
y_test = model(x_test)
print('y_test=', y_test.data)
5.逻辑回归求解二分类问题
知识点:sigmoid
激活函数,交叉熵损失函数BCELoss
问题描述:针对于输出为离散的分类问题
主要思路:利用sigmoid
函数求解某一类的概率,利用交叉熵损失函数BCELoss
衡量不同分布的差异大小
代码实现:在模型定义上最后一层添加sigmoid
函数,更改损失函数
import torch
import torch.functional as F
# 1.数据集处理
x_data = torch.tensor([[1.0], [2.0], [3.0]])
y_data = torch.tensor([[0.0], [0.0], [1.0]]) # 输出也要是float类型,不然损失值计算会报错
# 2.模型建立
class LogisticModual(torch.nn.Module):
def __init__(self):
super(LogisticModual,self).__init__()
self.linear = torch.nn.Linear(1, 1)
def forward(self, x):
y_pred = torch.sigmoid(self.linear(x)) # 在前馈过程中加入sigmoid函数将输出映射到01区间
return y_pred
model = LogisticModual()
# 3.损失函数和优化器
criterion = torch.nn.BCELoss() # 换用BCE交叉熵损失函数(适合二分类任务)
optimizer = torch.optim.SGD(model.parameters(), lr=0.1)
# 4.训练循环
for epoch in range(500):
y_pred = model(x_data)
loss = criterion(y_pred, y_data)
optimizer.zero_grad() # 梯度清零
loss.backward() # 反向传递
optimizer.step() # 梯度更新
print('epoch=', epoch, 'loss=', loss.item()) # 在print时注意是否为张量,取数值加item()
x_test = torch.tensor([[4.0]])
y_test = model(x_test)
print('y_test=', y_test.data.item())
# 绘图
import matplotlib.pyplot as plt
import numpy as np
x = np.linspace(0, 10, 200) # 0到10取200个点
x_t = torch.tensor(x, dtype=torch.float).view(200, 1) # 类似于numpy中的reshape
y_t = model(x_t)
plt.plot(x_t, y_t.data.numpy()) # tensor取数值转化到numpy
plt.ylabel('Probability of Pass')
plt.xlabel('Hours')
plt.plot([0,10], [0.5,0.5], c='r')
plt.grid()
plt.show()
注意输入和标签的数据集
绘图时先用np生成矩阵,利用torch.tensor转张量后利用view更改维度后输入网络
绘图时还要采用np形式,注意用tensor的data,直接接numpy()就能转为numpy数组了
6.多维数据分类问题
知识点:多层网络,激活函数连接
问题描述:输入数据存在多维特征,输出为多类别
主要思路:输入维度增加,基于矩阵乘法的行列关系,确定模型层输入输出维度即可。此外可以提高模型的复杂度,由单层网络变成多层网络,连接处需要加入激活函数引入非线性
代码实现:
- 数据集处理:利用
np.loadtxt()
确定分隔符,以np.float32
数据类型读入,转张量后切片分割数据集 - 在模型构造时设置多个线性层扩充网络层数,在前向传递的过程中每层接入
sigmoid
激活函数
import numpy as np
import torch
import matplotlib.pyplot as plt
# 1.数据准备(读取gz文件夹数据,并用逗号分隔)
xy = np.loadtxt('diabetes.csv.gz', delimiter=',', dtype=np.float32)
x_data = torch.from_numpy(xy[:, :-1]) # torch.Size([759, 8])
y_data = torch.from_numpy(xy[:, [-1]]) # torch.Size([759, 1])
# 2.模型构造
class Module(torch.nn.Module):
def __init__(self):
super(Module, self).__init__()
self.linear1 = torch.nn.Linear(8,6)
self.linear2 = torch.nn.Linear(6, 4)
self.linear3 = torch.nn.Linear(4, 1)
self.sigmoid = torch.nn.Sigmoid()
def forward(self, x):
x = self.sigmoid(self.linear1(x))
x = self.sigmoid(self.linear2(x))
x = self.sigmoid(self.linear3(x))
return x
model = Module()
# 3.构造损失函数和优化器
criterion = torch.nn.BCELoss(reduction='mean')
optimizer = torch.optim.SGD(model.parameters(), lr=0.01)
# 4.训练循环
epochlist = []
losslist = []
for epoch in range(100):
y_pred = model(x_data)
loss = criterion(y_pred, y_data)
epochlist.append(epoch)
losslist.append(loss.item())
print('epoch=', epoch, 'loss=', loss.item())
optimizer.zero_grad()
loss.backward()
optimizer.step()
plt.plot(epochlist, losslist)
plt.xlabel('epoch')
plt.ylabel('loss')
plt.show()
torch.from_numpy()可以将数据转为torch张量(送入网络前要先转为张量)
np.loadtxt()可以读取指定文件,同时支持压缩文件,包括gz,bz类型
注意输入输出的维度大小
7.数据集构建
知识点:Dataset
和Dataload
,以及minibatch
训练
问题描述:将数据集进行统一的封装
主要思路:利用Dataset
划分数据集,转tensor
张量,并支持索引;利用Dataload
进行批处理并打乱数据集
代码实现:
- 由于
Dataset
为抽象类,需要自定义后继承__init__
:一般根据传入的文件名去指定文件获取数据集,并定义len(shape后取行数), x, y作为属性__getitem__
:利用索引直接返回即可__len__
:直接返回数据长度的属性
- 而
Dataload
对其进行实例化即可,需要指定:数据集、是否打乱、多线程等(由于win
和linux
下多线程调用库不一致,因此主循环需要加入判断语句) - 由于采用batch进行训练,因此要进行嵌套for循环,
Dataload
处理成一批批数据了,每次读入一批
import numpy as np
import torch
from torch.utils.data import Dataset, DataLoader
import matplotlib.pyplot as plt
# 1.数据准备(Dataset为抽象类,需要实例化)
class DiabetesDataset(Dataset):
def __init__(self, filepath):
xy = np.loadtxt(filepath, delimiter=',', dtype=np.float32)
self.len = xy.shape[0]
self.x = torch.from_numpy(xy[:, :-1])
self.y = torch.from_numpy(xy[:, [-1]])
def __getitem__(self, index):
return self.x[index], self.y[index]
def __len__(self):
return self.len
dataset = DiabetesDataset('diabetes.csv.gz')
train_loader = DataLoader(dataset=dataset, batch_size=32, shuffle=True, num_workers=0)
# 2.模型定义
class LogicModule(torch.nn.Module):
def __init__(self):
super(LogicModule, self).__init__()
self.linear1 = torch.nn.Linear(8, 6)
self.linear2 = torch.nn.Linear(6, 4)
self.linear3 = torch.nn.Linear(4, 1)
self.sigmoid = torch.nn.Sigmoid()
def forward(self, x):
x = self.sigmoid(self.linear1(x))
x = self.sigmoid(self.linear2(x))
x = self.sigmoid(self.linear3(x))
return x
model = LogicModule()
# 3.损失函数和优化器
criterion = torch.nn.BCELoss()
optimizer = torch.optim.SGD(model.parameters(), lr=0.01)
# 4.模型训练(由于new_work在win和linux下不兼容,需要用if)
if __name__ == '__main__':
loss_list = []
for epoch in range(500):
loss_epoch = 0
for i, (x, y) in enumerate(train_loader): # 这里只有训练数据
y_pre = model(x)
loss = criterion(y_pre, y)
loss_epoch += loss.item()
# print('epoch=', epoch, 'loss=', loss.item())
optimizer.zero_grad()
loss.backward()
optimizer.step()
loss_list.append(loss_epoch/i)
# 绘制每个epoch中每个batch平均损失随epoch的图像
plt.plot(loss_list)
plt.show()
8.基于SoftMax的多分类
知识点:SoftMax
激活函数,多分类交叉熵CrossEntropyLoss
,图像transform
预处理,训练测试单独封装,``torchvision`库
问题描述:dataset
中MINIST
数据集读取方式为PIL
不能直接输入网络,同时多分类问题输出需要为一个分布
主要思路:利用transform
进行预处理,将图片拉直后进入线性网络,利用多分类交叉熵CrossEntropyLoss
计算损失值(线性层传入即可,CrossEntropyLoss = SoftMax + NLLLoss
)
代码实现:
- 数据处理:
transforms
构建Compose
时用ToTensor
转为张量,用Normalize
进行标准化(涉及到的函数都来自于torchvision.transforms
)- 利用
dataset
中MINIST
数据集指定下载、路径、是否为训练集、transform
- 利用
Dataloader
整理成分批数据集
- 利用
- 由于用线性网络实现,前向传播时先利用
view
将图像拉直过线性层(注意最后一层不用过激活函数),采用CrossEntropyLoss
损失函数 - 训练:内层循环单独拿出来(传入
epoch
)记录batch
数,进行输出(loss
存的是item()
,同时输出后记得置零) - 测试:
- 不用计算梯度,利用
with torch.no_grad():
- 利用正确数除以总数记录精确度
- 总数:累加每个
batch
的size(0)
- 正确数:利用``torch.max
找到最大概率值,获取其索引于真实值进行比较,利用
sum()`汇总数据
- 总数:累加每个
- 不用计算梯度,利用
import torch
from torchvision import transforms
from torchvision import datasets
from torch.utils.data import DataLoader
import torch.nn.functional as F
# 1.数据集准备
batch_size = 64
# 转化为tensor张量+归一化处理(均值、方差)
transform = transforms.Compose([transforms.ToTensor(), transforms.Normalize((0.1307,), (0.3081,))])
train_dataset = datasets.MNIST(root='../dataset/minist/', train=True, download=True, transform=transform)
train_loader = DataLoader(train_dataset, batch_size=batch_size, shuffle=True)
test_dataset = datasets.MNIST(root='../dataset/minist/', train=False, download=True, transform=transform)
test_loader = DataLoader(test_dataset, batch_size=batch_size, shuffle=False)
# 2.模型建立
class Net(torch.nn.Module):
def __init__(self):
super(Net, self).__init__()
self.l1 = torch.nn.Linear(784, 512)
self.l2 = torch.nn.Linear(512, 256)
self.l3 = torch.nn.Linear(256, 128)
self.l4 = torch.nn.Linear(128, 64)
self.l5 = torch.nn.Linear(64, 10)
def forward(self, x):
# 28*28 图片拉成1*784的向量(view(-1,b)表示在a参数未知情况下自动补齐行向量长度)
x = x.view(-1, 784)
x = F.relu(self.l1(x))
x = F.relu(self.l2(x))
x = F.relu(self.l3(x))
x = F.relu(self.l4(x))
return self.l5(x) # 后续要用到交叉熵,不用再过激活函数了
model = Net()
# 3.损失函数和优化器
criterion = torch.nn.CrossEntropyLoss()
optimizer = torch.optim.SGD(model.parameters(), lr=0.01, momentum=0.5) # 带冲量的SGD
# 4.单独把训练层拿出来
def train(epoch):
running_loss = 0.0
for batch_idx, (inputs, target) in enumerate(train_loader):
outputs = model(inputs)
loss = criterion(outputs, target)
running_loss += loss.item()
optimizer.zero_grad()
loss.backward()
optimizer.step()
if batch_idx % 300 == 299:
print('[%d, %5d] loss:%.3f', epoch+1, batch_idx+1, running_loss/300)
running_loss = 0.0
def test():
correct = 0
total = 0
with torch.no_grad(): # 不计算梯度
for data in test_loader:
images, labels = data
outputs = model(images)
# 取每行(十类)概率最大项,列为0,行为1
# (max输出两个张量,第一个是该维度最大值,第二个是序号),在此取第二个就行
_, predicted = torch.max(outputs.data, dim=1)
total += labels.size(0) # size获取了N*1,第一项可以读出总样本数
correct += (predicted == labels).sum().item() # 逐个元素比较后求总和
print('accuracy on test set: %d %%' % (100*correct/total))
# 为避免多进程错误
if __name__ == '__main__':
for epoch in range(10):
train(epoch)
test()
由于 transforms.Compose() 接受的参数是一个列表,所以需要使用 [] 将变换组织成一个列表,以便传递给这个函数。
输入线性层之前一定要记得拉直成向量
torch.max(dim=1)会返回两个值(数和索引),dim表示计算最值的维度,1表示列,也就是取每行的所有列中的最值
进行计数使用tensor.data
9. CNN基础篇
9.1 CNN
知识点:卷积层、池化层(主要在模型构建上)
问题描述:卷积神经网络保留图像空间结构,采用特征提取+分类的形式更有效处理图像
主要思路:利用卷积层+池化层叠加(池化层只改变图像大小)最后拉直向量过全连接层
代码实现:
- 在模型初始化定义好卷积层、池化层以及线性层(确定好输入输出维度)
- 前者只考虑通道数即可(卷积核的通道数要与输入一致,输出的通道数要和卷积核个数一致)
- 输入线性层前需要算好拉直后的维度,可以预设个输入,查看输出的
shape
- 前向传播
- 计算输入的
batch
,方便后续进行拉直x = x.view(batch_size, -1)
- 最后记得通过线性层+采用
CrossEntropyLoss()
- 计算输入的
import torch
from torchvision import transforms
from torchvision import datasets
from torch.utils.data import DataLoader
# 1.数据集准备
batch_size = 64
transform = transforms.Compose([transforms.ToTensor(), transforms.Normalize((0.1307,), (0.3081,))])
train_dataset = datasets.MNIST(root='../dataset/minist/', train=True, download=True, transform=transform)
train_loader = DataLoader(train_dataset, shuffle=True, batch_size=batch_size, num_workers=2)
test_dataset = datasets.MNIST(root='../dataset/minist/', train=False, download=True, transform=transform)
test_loader = DataLoader(test_dataset, shuffle=False, batch_size=batch_size, num_workers=2)
# 2.模型构建
class CNNNet(torch.nn.Module):
def __init__(self):
super(CNNNet, self).__init__()
self.cov1 = torch.nn.Conv2d(1, 10,5)
self.cov2 = torch.nn.Conv2d(10, 20,5)
self.pool = torch.nn.MaxPool2d(2)
self.linear = torch.nn.Linear(320, 10)
def forward(self, x):
batch_size = x.size(0) # B,C,W,H(之前的transform已经将图像转为tensor张量了,取第一个维度就是batch)
x = self.pool(torch.relu(self.cov1(x)))
x = self.pool(torch.relu(self.cov2(x)))
x = x.view(batch_size, -1) # batch是不变的,把CWH拉长
# print(x.shape)
return self.linear(x)
model = CNNNet()
# sample_image = torch.randn(1, 1, 28, 28) # 1张图像,1个通道,28x28大小的图像
# output = model(sample_image)
# 3.损失值和优化器
criterion = torch.nn.CrossEntropyLoss()
optimizer = torch.optim.SGD(model.parameters(), lr=0.01, momentum=0.5)
# 4.训练循环
def train(epoch):
l = 0.0
for batch_index, (x, y) in enumerate(train_loader):
y_pred = model(x)
optimizer.zero_grad()
loss = criterion(y_pred, y)
l += loss.item()
loss.backward()
optimizer.step()
if batch_index % 300 == 299:
print(f'[epoch{epoch+1}---------batch={batch_index+1}---------loss={round(100*l/300, 3)}]')
l = 0.0 # 输出完记得置为0
def test():
size = 0
acc = 0
with torch.no_grad():
for (x, y) in test_loader:
y_pred = model(x)
_, predict = torch.max(y_pred.data, dim=1) # 0列1行,注意这里取的是data(用到张量的时候要格外小心)
size += predict.size(0)
acc += (predict == y).sum().item() # 与标签进行比较
print('test accuracy= %.3f %%' % (100 * acc / size))
if __name__ == "__main__":
for epoch in range(10):
train(epoch)
test()
9.2 CNN(GPU加速)
知识点:将模型和数据上载到GPU进行加速
代码实现:设置device
,利用to()
对模型和数据上载即可
import torch
from torchvision import transforms
from torchvision import datasets
from torch.utils.data import DataLoader
import matplotlib.pyplot as plt
# 1.数据集准备
batch_size = 64
transform = transforms.Compose([transforms.ToTensor(), transforms.Normalize((0.1307,), (0.3081,))])
train_dataset = datasets.MNIST(root='../dataset/minist/', train=True, download=True, transform=transform)
train_loader = DataLoader(train_dataset, shuffle=True, batch_size=batch_size, num_workers=2)
test_dataset = datasets.MNIST(root='../dataset/minist/', train=False, download=True, transform=transform)
test_loader = DataLoader(test_dataset, shuffle=False, batch_size=batch_size, num_workers=2)
# 2.模型构建
class CNNNet(torch.nn.Module):
def __init__(self):
super(CNNNet, self).__init__()
self.cov1 = torch.nn.Conv2d(1, 10,5)
self.cov2 = torch.nn.Conv2d(10, 20,5)
self.pool = torch.nn.MaxPool2d(2)
self.linear = torch.nn.Linear(320, 10)
def forward(self, x):
batch_size = x.size(0) # B,C,W,H(之前的transform已经将图像转为tensor张量了,取第一个维度就是batch)
x = self.pool(torch.relu(self.cov1(x)))
x = self.pool(torch.relu(self.cov2(x)))
x = x.view(batch_size, -1) # batch是不变的,把CWH拉长
return self.linear(x)
model = CNNNet()
device = torch.device('cuda'if torch.cuda.is_available() else 'cpu')
model.to(device)
# 3.损失值和优化器
criterion = torch.nn.CrossEntropyLoss()
optimizer = torch.optim.SGD(model.parameters(), lr=0.01, momentum=0.5)
# 4.训练循环
def train(epoch):
l = 0.0
for batch_index, (x, y) in enumerate(train_loader):
x, y = x.to(device), y.to(device)
y_pred = model(x)
optimizer.zero_grad()
loss = criterion(y_pred, y)
l += loss.item()
loss.backward()
optimizer.step()
if batch_index % 300 == 299:
print(f'[epoch{epoch+1}---------batch={batch_index+1}---------loss={round(100*l/300, 3)}]')
l = 0.0 # 输出完记得置为0
def test():
size = 0
acc = 0
with torch.no_grad():
for (x, y) in test_loader:
x, y = x.to(device), y.to(device)
y_pred = model(x)
_, predict = torch.max(y_pred.data, dim=1) # 0列1行,注意这里取的是data(用到张量的时候要格外小心)
size += predict.size(0)
acc += (predict == y).sum().item() # 与标签进行比较
print('test accuracy= %.3f %%' % (100 * acc / size))
return acc / size
if __name__ == "__main__":
epoch_list = []
acc_list = []
for epoch in range(10):
train(epoch)
acc = test()
epoch_list.append(epoch)
acc_list.append(acc)
plt.plot(epoch_list, acc_list)
plt.ylabel('accuracy')
plt.xlabel('epoch')
plt.show()
10. CNN(高级篇)
10.1 GoogleNet(Inception层)
知识点:Inception
层模块单独定义
问题描述:造神经网络里面有一些超参数比较难选(例如卷积核大小)
主要思路:利用多个分支设置不同的卷积核大小,让模型自己进行选择(哪个好就赋较大权重)
代码实现:
- 定于
InceptionA
模块,由于要进行复用,将输入通道作为参数传递进来 - 涉及不同的分支,整理成列表,在前向通道利用
torch.cat()
进行拼接即可,指定dim=1,以C维度进行拼接
import torch
from torchvision import datasets
from torch.utils.data import DataLoader
from torchvision import transforms
import torch.nn.functional as F
# 1.数据集准备
batch_size = 64
transform = transforms.Compose([transforms.ToTensor(), transforms.Normalize((0.1307,), (0.3081,))])
train_dataset = datasets.MNIST(root='../dataset/minist', train = True, download=True, transform=transform)
train_loader = DataLoader(train_dataset, shuffle=True, batch_size=batch_size, num_workers=2)
test_dataset = datasets.MNIST(root='../dataset/minist', train = False, download=True, transform=transform)
test_loader = DataLoader(test_dataset, shuffle=False, batch_size=batch_size, num_workers=2)
# 2.模型构建
class InceptionA(torch.nn.Module):
def __init__(self, in_channel):
super(InceptionA,self).__init__()
# 1*1卷积分支
self.branch1x1 = torch.nn.Conv2d(in_channel, 16, 1)
# 5*5卷积分支
self.branch5x5_1 = torch.nn.Conv2d(in_channel, 16, 1)
self.branch5x5_2 = torch.nn.Conv2d(16, 24, 5, padding=2)
# 3*3卷积分支
self.branch3x3_1 = torch.nn.Conv2d(in_channel, 16, 1)
self.branch3x3_2 = torch.nn.Conv2d(16,24, 3, padding=1)
self.branch3x3_3 = torch.nn.Conv2d(24, 24, 3, padding=1)
# 平均池化分支
self.pool = torch.nn.Conv2d(in_channel, 24, 1)
def forward(self, x):
branch1x1 = self.branch1x1(x)
branch5x5 = self.branch5x5_1(x)
branch5x5 = self.branch5x5_2(branch5x5)
branch3x3 = self.branch3x3_1(x)
branch3x3 = self.branch3x3_2(branch3x3)
branch3x3 = self.branch3x3_3(branch3x3)
branch_pool = F.avg_pool2d(x, kernel_size=3, stride=1, padding=1)
branch_pool = self.pool(branch_pool)
outputs = [branch1x1, branch5x5, branch3x3, branch_pool]
return torch.cat(outputs, dim=1) # b,c,w,h c对应的是dim=1
class Net(torch.nn.Module):
def __init__(self):
super(Net, self).__init__()
self.c1 = torch.nn.Conv2d(1, 10, 5)
self.c2 = torch.nn.Conv2d(88, 20, 5)
self.inception1 = InceptionA(10) # 与conv1 中的10对应
self.inception2 = InceptionA(20) # 与conv2 中的20对应
self.mp = torch.nn.MaxPool2d(2) # 图像缩小一半 12 (不要改步长啊)
self.linear = torch.nn.Linear(1408, 10)
def forward(self, x):
batch = x.size(0)
x = torch.relu(self.mp(self.c1(x))) # b*10*12*12
x = self.inception1(x)
x = torch.relu(self.mp(self.c2(x))) # b*20*4*4
x = self.inception2(x)
x = x.view(batch, -1)
# print(x.shape)
return self.linear(x)
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
model = Net().to(device)
# # 创建一个示例图像查看模型输出shape(好给全连接层赋值)----->输出:torch.Size([1, 1408])
# sample_image = torch.randn(1, 1, 28, 28) # 1张图像,1个通道,28x28大小的图像
# output = model(sample_image)
# 3.损失函数和优化器
criterion = torch.nn.CrossEntropyLoss()
optimizer = torch.optim.SGD(model.parameters(), lr=0.01)
# 4.训练
def train(epoch):
running_loss = 0.0
for batch_idex, (x, y) in enumerate(train_loader):
x, y = x.to(device), y.to(device)
y_pred = model(x)
optimizer.zero_grad()
loss = criterion(y_pred, y)
running_loss += loss.item()
loss.backward()
optimizer.step()
if batch_idex % 300 == 299:
print(f'epoch{epoch+1}--------batch{batch_idex+1}-------loss={round(running_loss/300, 3)}')
running_loss = 0.0
def test():
total = 0
acc = 0
with torch.no_grad():
for (x, y) in test_loader:
x, y = x.to(device), y.to(device)
y_pred = model(x)
total += y_pred.size(0)
_, predicted = torch.max(y_pred, dim=1)
acc += (predicted == y).sum().item()
print('test= %.3f %%' % (100 * acc/total))
if __name__ == '__main__':
for epoch in range(10):
train(epoch)
test()
10.2 Residual Net
知识点:残差层定义
问题描述:卷积核层数不是越深越好,可能存在梯度消失
主要思路:引入残差连接,拼接后再激活,计算梯度的时候就能有所保留,要求输入输出大小相同
代码实现:定义残差块类,指定输入通道数,跳转拼接后再激活。模型构建时再定义相关层
import torch
from torchvision import datasets
from torch.utils.data import DataLoader
from torchvision import transforms
import torch.nn.functional as F
# 1.数据集准备
batch_size = 64
transform = transforms.Compose([transforms.ToTensor(), transforms.Normalize((0.1307,), (0.3081,))])
train_dataset = datasets.MNIST(root='../dataset/minist', train = True, download=True, transform=transform)
train_loader = DataLoader(train_dataset, shuffle=True, batch_size=batch_size, num_workers=2)
test_dataset = datasets.MNIST(root='../dataset/minist', train = False, download=True, transform=transform)
test_loader = DataLoader(test_dataset, shuffle=False, batch_size=batch_size, num_workers=2)
# 2.模型构建
class ResidualBlock(torch.nn.Module):
def __init__(self, channels):
super(ResidualBlock,self).__init__()
self.channels = channels # 过残差连接输入输出通道不变
self.conv1 = torch.nn.Conv2d(channels, channels, 3,padding=1)
self.conv2 = torch.nn.Conv2d(channels, channels, 3,padding=1)
def forward(self, x):
y = F.relu(self.conv1(x))
y = self.conv2(y)
return F.relu(x + y) # 先求和再激活
class Net(torch.nn.Module):
def __init__(self):
super(Net, self).__init__()
self.c1 = torch.nn.Conv2d(1, 16, 5)
self.c2 = torch.nn.Conv2d(16, 32, 5)
self.rblock1 = ResidualBlock(16) # 与conv1 中的16对应
self.rblock2 = ResidualBlock(32) # 与conv2 中的32对应
self.mp = torch.nn.MaxPool2d(2) # 图像缩小一半 12 (不要改步长啊)
self.l = torch.nn.Linear(512, 10)
def forward(self, x):
batch = x.size(0)
x = torch.relu(self.mp(self.c1(x))) # b*10*12*12
x = self.rblock1(x)
x = torch.relu(self.mp(self.c2(x))) # b*20*4*4
x = self.rblock2(x)
x = x.view(batch, -1)
# print(x.shape)
return self.l(x)
device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')
model = Net().to(device)
# # 创建一个示例图像查看模型输出shape(好给全连接层赋值)----->输出:torch.Size([1, 1408])
# sample_image = torch.randn(1, 1, 28, 28) # 1张图像,1个通道,28x28大小的图像
# output = model(sample_image)
# 3.损失函数和优化器
criterion = torch.nn.CrossEntropyLoss()
optimizer = torch.optim.SGD(model.parameters(), lr=0.01)
# 4.训练
def train(epoch):
running_loss = 0.0
for batch_idex, (x, y) in enumerate(train_loader):
x, y = x.to(device), y.to(device)
y_pred = model(x)
optimizer.zero_grad()
loss = criterion(y_pred, y)
running_loss += loss.item()
loss.backward()
optimizer.step()
if batch_idex % 300 == 299:
print(f'epoch{epoch+1}--------batch{batch_idex+1}-------loss={round(running_loss/300, 3)}')
running_loss = 0.0
def test():
total = 0
acc = 0
with torch.no_grad():
for (x, y) in test_loader:
x, y = x.to(device), y.to(device)
y_pred = model(x)
total += y_pred.size(0)
_, predicted = torch.max(y_pred, dim=1)
acc += (predicted == y).sum().item()
print('test= %.3f %%' % (100 * acc/total))
if __name__ == '__main__':
for epoch in range(10):
train(epoch)
test()
11. RNN循环神经网络
知识点:RNN处理数据维度,两种构建RNN的方法
问题描述:对于序列数据采用循环神经网络
11.1 RNNCell
主要内容:随机生成seq_size个句子,通过h0
为全零的RNN网络,探究输入输出以及隐层的维度
代码实现:利用RNNCell
处理单个运算,RNNCell只是RNN的一个单元,用于处理一个时间步的输入数据,需要在循环中手动处理时间步。
seqLen:每个句子所含词的个数
batchSize:每次处理词的批数
InputSize:每个词嵌入向量的位数
dataset: (seq_size, batch_size, input_size)——>整个数据集维度
input: (batch_size, input_size)——>每个时间步处理的输入(input in dataset)
hidden: (batch_size, output_size)——>每次处理的隐层(即为output)
import torch
batch_size = 1
seq_len = 3
input_size = 4
hidden_size = 2
# Construction of RNNCell
cell = torch.nn.RNNCell(input_size=input_size, hidden_size=hidden_size)
# Wrapping the sequence into:(seqLen,batchSize,InputSize)
dataset = torch.randn(seq_len, batch_size, input_size) # (3,1,4)
# Initializing the hidden to zero
hidden = torch.zeros(batch_size, hidden_size) # (1,2)
for idx, input in enumerate(dataset):
print('=' * 20, idx, '=' * 20)
print('Input size:', input.shape) # (batch_size, input_size)
# 按序列依次输入到cell中,seq_len=3,故循环3次
hidden = cell(input, hidden) # 返回的hidden是下一次的输入之一,循环使用同一个cell
print('output size:', hidden.shape) # (batch_size, hidden_size)
print(hidden.data)
11.2 RNNCell
主要内容:随机生成seq_size个句子,通过h0为全零的RNN网络,探究输入输出以及隐层的维度
代码实现:
输入dataset:整个序列(seq_size, batch_size, input_size) 取每一个词input(batch_size, input_size)
隐层:(num_layers, batch_size, hidden_size)
输出(有两个): output:h1~hn (seq_size, batch_size, hidden_size)
hidden: hn (num_layers, batch_size, hidden_size)
import torch
batch_size = 1
seq_len = 3
input_size = 4
hidden_size = 2
num_layers = 1 # RNN层数
# 构建RNNCell
rnn = torch.nn.RNN(input_size=input_size, hidden_size=hidden_size, num_layers=num_layers)
# 构建inputs, 需要整理成(seq_len, batch_size, input_size)
inputs = torch.randn(seq_len, batch_size, input_size)
# 全零初始化h0,需要整理成(num_layers, batch_size, hidden_size)
hidden = torch.zeros(num_layers, batch_size, hidden_size)
output, hidden = rnn(inputs, hidden)
print('Output size:', output.shape) # (seq_len, batch_size, hidden_size)
print('Output:', output.data)
print('Hidden size:', hidden.shape) # (num_layers, batch_size, hidden_size)
print('Hidden:', hidden.data)
1.与Cell相比多了一个num_layers,同时不用自己写循环
2.输出output是历史hidden记录, hidden为最后一个隐层输出
3.可以指定batch_first参数,使得以(batch_size, seq_len, input_size)进行输入
两者的区别
1.RNNCell只是RNN的一个单元,用于处理一个时间步的输入数据,需要在循环中手动处理时间步。
2.完整的RNN模型,它内部包含了循环结构,可以一次性处理整个序列的输入,从而避免了手动处理时间步的繁琐过程。
11.3 基于RNNCell处理seq->seq任务
import torch
# 1.参数设置
input_size = 4
hidden_size = 4 # 输出维度要与输入一致,才能在同一个表上查询
batch_size = 1
# 2.数据准备
index2char = ['e', 'h', 'l', 'o'] # 字典
x_data = [1, 0, 2, 2, 3] # hello
y_data = [3, 1, 2, 3, 2] # 标签:ohlol
# one-hot 查找表(维度为字典中词的个数)
one_hot_lookup = [[1, 0, 0, 0], # 用来将x_data转换为one-hot向量的参照表
[0, 1, 0, 0],
[0, 0, 1, 0],
[0, 0, 0, 1]]
x_one_hot = [one_hot_lookup[x] for x in x_data] # 转化为独热向量,维度(seq, batch, input)
dataset = torch.Tensor(x_one_hot).view(-1, batch_size, input_size) # 转tensor后整理成(𝒔𝒆𝒒𝑳𝒆𝒏,𝒃𝒂𝒕𝒄𝒉𝑺𝒊𝒛𝒆,𝒊𝒏𝒑𝒖𝒕𝑺𝒊𝒛𝒆)
labels = torch.LongTensor(y_data).view(-1, 1) # 每层对应一个类(seqLen,𝟏)计算交叉熵损失时标签不需要我们进行one-hot编码,其内部会自动进行处理
# 3.模型构建
class Model(torch.nn.Module):
def __init__(self, input_size, hidden_size, batch_size): # 需要指定输入,隐层,批
super(Model, self).__init__()
self.batch_size = batch_size
self.input_size = input_size
self.hidden_size = hidden_size
self.rnncell = torch.nn.RNNCell(input_size=self.input_size, hidden_size=hidden_size)
def forward(self, input, hidden):
hidden =self.rnncell(input, hidden)
return hidden
def init_hidden(self): # 只有在初始化隐藏层需要batch_size
return torch.zeros(self.batch_size, self.hidden_size)
net = Model(input_size, hidden_size, batch_size)
# 4.损失和优化器
criterion = torch.nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(net.parameters(), lr=0.1)
# 5.训练
for epoch in range(15):
loss = 0
optimizer.zero_grad()
hidden = net.init_hidden() # 初始化h0
print('Predicted string:', end='')
for input, label in zip(dataset, labels):
hidden = net(input, hidden)
loss += criterion(hidden, label) # 这里要求总损失进行梯度下降(构建计算图),因此不能取item()
_, idx = hidden.max(dim=1) # 取出概率最大的索引
print(index2char[idx.item()], end='')
loss.backward()
optimizer.step()
print(',Epoch [%d / 15] loss:%.4f' % (epoch+1, loss.item()))
对于输入序列进行独热编码,构建RNNCell,对每次输入的字符依次进入循环
针对于分类任务的输出要使用LongTensor类型
求总损失进行梯度下降(构建计算图),因此不能取item()
11.4 基于RNN处理seq->seq任务
import torch
# 1.参数设置
seq_len = 5
input_size = 4
hidden_size = 4
batch_size = 1
# 2.数据准备
index2char = ['e', 'h', 'l', 'o']
x_data = [1, 0, 2, 2, 3]
y_data = [3, 1, 2, 3, 2]
one_hot_lookup = [[1, 0, 0, 0],
[0, 1, 0, 0],
[0, 0, 1, 0],
[0, 0, 0, 1]]
x_one_hot = [one_hot_lookup[x] for x in x_data]
inputs = torch.Tensor(x_one_hot).view(-1, batch_size, input_size)
labels = torch.LongTensor(y_data)
# 3.模型构建
class Model(torch.nn.Module):
def __init__(self, input_size, hidden_size, batch_size, num_layers=1): # 需要指定输入,隐层,批
super(Model, self).__init__()
self.batch_size = batch_size
self.input_size = input_size
self.hidden_size = hidden_size
self.num_layers = num_layers
self.rnn = torch.nn.RNN(input_size=self.input_size, hidden_size=self.hidden_size, num_layers=self.num_layers)
def forward(self, input):
hidden = torch.zeros(self.num_layers,
self.batch_size,
self.hidden_size)
out, _ = self.rnn(input, hidden) # out: tensor of shape (seq_len, batch, hidden_size)
return out.view(-1, self.hidden_size) # 将输出的三维张量转换为二维张量,(𝒔𝒆𝒒𝑳𝒆𝒏×𝒃𝒂𝒕𝒄𝒉𝑺𝒊𝒛𝒆,𝒉𝒊𝒅𝒅𝒆𝒏𝑺𝒊𝒛𝒆)
net = Model(input_size, hidden_size, batch_size)
# 4.损失和优化器
criterion = torch.nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(net.parameters(), lr=0.5)
# 5.训练
for epoch in range(15):
optimizer.zero_grad()
outputs = net(inputs)
print(outputs.shape)
loss = criterion(outputs, labels)
loss.backward()
optimizer.step()
_, idx = outputs.max(dim=1)
idx = idx.data.numpy()
print('Predicted string: ', ''.join([index2char[x] for x in idx]), end='')
print(',Epoch [%d / 15] loss:%.4f' % (epoch+1, loss.item()))
输入:seq_size * batch_size * input_size
输出:seq_size * batch_size * hidden_size, 利用view变成(seq_size * batch_size, hidden_size)作为一个矩阵
labels: seq_size * batch_size * 1 (序列中每一个样本的分类)
最后的out和labels是针对整个seq进行比较计算损失的
11.5 含嵌入层的RNN网络
import torch
# 1、确定参数
num_class = 4
input_size = 4
hidden_size = 8
embedding_size = 10
num_layers = 2
batch_size = 1
seq_len = 5
# 2、准备数据
index2char = ['e', 'h', 'l', 'o'] # 字典
x_data = [[1, 0, 2, 2, 3]] # (batch_size, seq_len) 用字典中的索引(数字)表示来表示hello
y_data = [3, 1, 2, 3, 2] # (batch_size * seq_len) 标签:ohlol
inputs = torch.LongTensor(x_data) # (batch_size, seq_len)
labels = torch.LongTensor(y_data) # (batch_size * seq_len)
# 3、构建模型
class Model(torch.nn.Module):
def __init__(self):
super(Model, self).__init__()
self.emb = torch.nn.Embedding(num_class, embedding_size)
self.rnn = torch.nn.RNN(input_size=embedding_size, hidden_size=hidden_size, num_layers=num_layers,
batch_first=True)
self.fc = torch.nn.Linear(hidden_size, num_class)
def forward(self, x):
hidden = torch.zeros(num_layers, x.size(0), hidden_size) # (num_layers, batch_size, hidden_size)
x = self.emb(x) # 返回(batch_size, seq_len, embedding_size)
x, _ = self.rnn(x, hidden) # 返回(batch_size, seq_len, hidden_size)
x = self.fc(x) # 返回(batch_size, seq_len, num_class)
return x.view(-1, num_class) # (batch_size * seq_len, num_class)
net = Model()
# 4、损失和优化器
criterion = torch.nn.CrossEntropyLoss()
optimizer = torch.optim.Adam(net.parameters(), lr=0.05) # Adam优化器
# 5、训练
for epoch in range(15):
optimizer.zero_grad()
outputs = net(inputs)
loss = criterion(outputs, labels)
loss.backward()
optimizer.step()
_, idx = outputs.max(dim=1)
idx = idx.data.numpy()
print('Predicted string: ', ''.join([index2char[x] for x in idx]), end='')
print(', Epoch [%d/15] loss: %.4f' % (epoch + 1, loss.item()))
11.6 RNN分类器实现人名国家对应
import csv
import time
import matplotlib.pyplot as plt
import numpy as np
import math
import gzip # 用于读取压缩文件
import torch
import torch.optim as optim
from torch.utils.data import Dataset, DataLoader
from torch.nn.utils.rnn import pack_padded_sequence
# 1.超参数设置
HIDDEN_SIZE = 100
BATCH_SIZE = 256
N_LAYER = 2 # RNN的层数
N_EPOCHS = 100
N_CHARS = 128 # ASCII码的个数
USE_GPU = False
# 工具类函数
# 把名字转换成ASCII码,返回ASCII码值列表和名字的长度
def name2list(name):
arr = [ord(c) for c in name]
return arr, len(arr)
# 是否把数据放到GPU上
def create_tensor(tensor):
if USE_GPU:
device = torch.device('cuda:0')
tensor = tensor.to(device)
return tensor
def timesince(since):
now = time.time()
s = now - since
m = math.floor(s / 60) # math.floor()向下取整
s -= m * 60
return '%dmin %ds' % (m, s) # 多少分钟多少秒
# 2.构建数据集
class NameDataset(Dataset):
def __init__(self, is_train=True):
# 文件读取
filename = '../dataset/names_train.csv.gz' if is_train else '../dataset/names_test.csv.gz'
with gzip.open(filename, 'rt') as f: # rt表示以只读模式打开文件,并将文件内容解析为文本形式
reader = csv.reader(f)
rows =list(reader) # 每个元素由一个名字和国家组成
# 提取属性
self.names = [row[0] for row in rows]
self.len = len(self.names)
self.countries = [row[1] for row in rows]
# 编码处理
self.country_list = list(sorted(set(self.countries))) # 列表,按字母表顺序排序,去重后有18个国家名
self.country_dict = self.get_countries_dict() # 字典,国家名对应序号标签
self.country_num = len(self.country_list)
def __getitem__(self, item):
# 索引获取
return self.names[item], self.country_dict[self.countries[item]] # 根据国家去字典查找索引
def __len__(self):
# 获取个数
return self.len
def get_countries_dict(self):
# 根据国家名对应序号
country_dict = dict()
for idx, country_name in enumerate(self.country_list):
country_dict[country_name] = idx
return country_dict
def idx2country(self, index):
# 根据索引返回国家名字
return self.country_list[index]
def get_countries_num(self):
# 返回国家名个数(分类的总个数)
return self.country_num
# 3.实例化数据集
train_set = NameDataset(is_train=True)
train_loader = DataLoader(train_set, shuffle=True, batch_size=BATCH_SIZE, num_workers=2)
test_set = NameDataset(is_train=False)
test_loder = DataLoader(test_set, shuffle=False, batch_size=BATCH_SIZE, num_workers=2)
N_COUNTRY = train_set.get_countries_num() # 18个国家名,即18个类别
# 4.模型构建
class GRUClassifier(torch.nn.Module):
def __init__(self, input_size, hidden_size, output_size, n_layers=1, bidirectional=True):
super(GRUClassifier, self).__init__()
self.hidden_size = hidden_size
self.n_layers = n_layers
self.n_directions = 2 if bidirectional else 1
# 词嵌入层,将词语映射到hidden维度
self.embedding = torch.nn.Embedding(input_size, hidden_size)
# GRU层(输入为特征数,这里是embedding_size,其大小等于hidden_size))
self.gru = torch.nn.GRU(hidden_size, hidden_size, num_layers=n_layers, bidirectional=bidirectional)
# 线性层
self.fc = torch.nn.Linear(hidden_size * self.n_directions, output_size)
def _init_hidden(self, bath_size):
# 初始化权重,(n_layers * num_directions 双向, batch_size, hidden_size)
hidden = torch.zeros(self.n_layers * self.n_directions, bath_size, self.hidden_size)
return create_tensor(hidden)
def forward(self, input, seq_lengths):
# 转置 B X S -> S X B
input = input.t() # 此时的维度为seq_len, batch_size
batch_size = input.size(1)
hidden = self._init_hidden(batch_size)
# 嵌入层处理 input:(seq_len,batch_size) -> embedding:(seq_len,batch_size,embedding_size)
embedding = self.embedding(input)
# 进行打包(不考虑0元素,提高运行速度)需要将嵌入数据按长度排好
gru_input = pack_padded_sequence(embedding, seq_lengths)
# output:(*, hidden_size * num_directions),*表示输入的形状(seq_len,batch_size)
# hidden:(num_layers * num_directions, batch, hidden_size)
output, hidden = self.gru(gru_input, hidden)
if self.n_directions == 2:
hidden_cat = torch.cat([hidden[-1], hidden[-2]], dim=1) # hidden[-1]的形状是(1,256,100),hidden[-2]的形状是(1,256,100),拼接后的形状是(1,256,200)
else:
hidden_cat = hidden[-1] # (1,256,100)
fc_output = self.fc(hidden_cat)
return fc_output
# 3.数据处理(姓名->数字)
def make_tensors(names, countries):
# 获取嵌入长度从大到小排序的seq_tensor(嵌入向量)、seq_lengths(对应长度)、countries(对应顺序的国家序号)-> 便于pack_padded_sequence处理
name_len_list = [name2list(name) for name in names] # 每个名字对应的1列表
name_seq = [sl[0] for sl in name_len_list] # 姓名列表
seq_lengths = torch.LongTensor([sl[1] for sl in name_len_list]) # 名字对应的字符个数
countries = countries.long() # PyTorch 中,张量的默认数据类型是浮点型 (float),这里转换成整型,可以避免浮点数比较时的精度误差,从而提高模型的训练效果
# 创建全零张量,再依次进行填充
# 创建了一个 len(name_seq) * seq_length.max()维的张量
seq_tensor = torch.zeros(len(name_seq), seq_lengths.max()).long()
for idx, (seq, seq_len) in enumerate(zip(name_seq, seq_lengths)):
seq_tensor[idx, :seq_len] = torch.LongTensor(seq)
# 为了使用pack_padded_sequence,需要按照长度排序
# perm_idx是排序后的数据在原数据中的索引,seq_tensor是排序后的数据,seq_lengths是排序后的数据的长度,countries是排序后的国家
seq_lengths, perm_idx = seq_lengths.sort(dim=0, descending=True) # descending=True 表示按降序进行排序,即从最长的序列到最短的序列。
seq_tensor = seq_tensor[perm_idx]
countries = countries[perm_idx]
return create_tensor(seq_tensor), create_tensor(seq_lengths), create_tensor(countries)
# 训练循环
def train(epoch, start):
total_loss = 0
for i, (names, countries) in enumerate(train_loader, 1):
inputs, seq_lengths, target = make_tensors(names, countries) # 输入、每个序列长度、输出
output = model(inputs, seq_lengths)
loss = criterion(output, target)
optimizer.zero_grad()
loss.backward()
optimizer.step()
total_loss += loss.item()
if i % 10 == 0:
print(f'[{timesince(start)}] Epoch {epoch} ', end='')
print(f'[{i * len(inputs)}/{len(train_set)}] ', end='')
print(f'loss={total_loss / (i * len(inputs))}') # 打印每个样本的平均损失
return total_loss
# 测试循环
def test():
correct = 0
total = len(test_set)
print('evaluating trained model ...')
with torch.no_grad():
for i, (names, countries) in enumerate(test_loder, 1):
inputs, seq_lengths, target = make_tensors(names, countries)
output = model(inputs, seq_lengths)
pred = output.max(dim=1, keepdim=True)[1] # 返回每一行中最大值的那个元素的索引,且keepdim=True,表示保持输出的二维特性
correct += pred.eq(target.view_as(pred)).sum().item() # 计算正确的个数
percent = '%.2f' % (100 * correct / total)
print(f'Test set: Accuracy {correct}/{total} {percent}%')
return correct / total # 返回的是准确率,0.几几的格式,用来画图
if __name__ == '__main__':
model = GRUClassifier(N_CHARS, HIDDEN_SIZE, N_COUNTRY, N_LAYER)
criterion = torch.nn.CrossEntropyLoss()
optimizer = optim.Adam(model.parameters(), lr=0.001)
device = 'cuda:0' if USE_GPU else 'cpu'
model.to(device)
start = time.time()
print('Training for %d epochs...' % N_EPOCHS)
acc_list = []
# 在每个epoch中,训练完一次就测试一次
for epoch in range(1, N_EPOCHS + 1):
# Train cycle
train(epoch, start)
acc = test()
acc_list.append(acc)
# 绘制在测试集上的准确率
epoch = np.arange(1, len(acc_list) + 1)
acc_list = np.array(acc_list)
plt.plot(epoch, acc_list)
plt.xlabel('Epoch')
plt.ylabel('Accuracy')
plt.grid()
plt.show()