自己学习的总结,有不对的地方欢迎指正,感谢各位
一、损失函数:
1.简介:
Loss Function : torch.nn — PyTorch 2.2 documentationhttps://pytorch.org/docs/stable/nn.html#loss-functions
本节中主要讲了:
nn.L1Loss、nn.MSELoss、 nn.CrossEntropyLoss(链接可以直接跳转)
2.代码:
# 损失函数与反向传播
import torch
from torch.nn import L1Loss
from torch import nn
inputs = torch.tensor([1, 2, 3], dtype=torch.float32)
targets = torch.tensor([1, 2, 5], dtype=torch.float32)
print("inputs: " + str(inputs.shape)) # 格式显示
print("targets: " + str(targets.shape))
inputs = torch.reshape(inputs, (1, 1, 1, 3)) # batch_size, channel, 行, 列
targets = torch.reshape(targets, (1, 1, 1, 3))
print("inputs_reshape: " + str(inputs.shape)) # 格式显示
print("inputs_reshape: " + str(inputs.shape))
loss = L1Loss() # 默认:reduction='sum' 均差
result = loss(inputs, targets)
print(result) # ( 0 + 0 + 2 )/3 = 0.6667
loss = L1Loss(reduction='sum') # 相加
result_sum = loss(inputs, targets)
print(result_sum) # ( 0 + 0 + 2 ) = 2
loss_mse = nn.MSELoss() # 均方差
result_mse = loss_mse(inputs, targets)
print(result_mse) # ( 0 + 0 + 2^2 )/3 = 1.333
# 交叉熵
""" 公式: Loss(x, class) = -x[class] + log{ ∑exp(x[j]) }"""
x = torch.tensor([0.1, 0.2, 0.3])
y = torch.tensor([1]) # 0.1, 0.2, 0.3 第0位、第1位、第2位、 1代表了0.2
x = torch.reshape(x, (1, 3)) # 转换成 batch_size, class
loss_cross = nn.CrossEntropyLoss() # 交叉熵
result_cross = loss_cross(x, y) # Loss(x, class) = -0.2 + log{ exp(0.1) + exp(0.2) + exp(0.3) } = 1.1019
print(result_cross)
3.运行结果:
二、反向传播:
1.作用:求出参数梯度,以便进行参数优化
2.代码:
# 如何在之前写的神经网络中使用 Loss Function ( 损失函数 )
# 1.计算实际输出和目标输出之间的差距
# 2.为我们更新输出提供一定依据( 反向传播 ) 可以求出参数梯度
import torchvision.datasets
from torch import nn
from torch.nn import Conv2d, MaxPool2d, Flatten, Linear, Sequential
from torch.utils.data import DataLoader
dataset = torchvision.datasets.CIFAR10("../data_conv2d", train=False, transform=torchvision.transforms.ToTensor(),
download=True)
dataloader = DataLoader(dataset, batch_size=1)
class Tudui(nn.Module):
def __init__(self):
super(Tudui, self).__init__()
self.model1 = Sequential(
Conv2d(3, 32, 5, padding=2),
MaxPool2d(2),
Conv2d(32, 32, 5, padding=2),
MaxPool2d(2),
Conv2d(32, 64, 5, padding=2),
MaxPool2d(2),
Flatten(),
Linear(1024, 64),
Linear(64, 10)
)
def forward(self, x):
x = self.model1(x)
return x
loss = nn.CrossEntropyLoss() # 公式:Loss(x, class) = -x[class] + log{ ∑exp(x[j]) }
tudui = Tudui()
for data in dataloader:
imgs, targets = data
outputs = tudui(imgs)
# print(outputs)
# print(targets)
result_loss = loss(outputs, targets) # 计算实际输出和目标输出之间的差距
# print(result_loss)
result_loss.backward()
print("ok")
3.结果:
在调试中运行,并按照如下依次打开:
其中的 grad 便是梯度,在单步执行此语句之后:
result_loss.backward()
grad会被改变:
三、优化器:
1.作用:减小loss
torch.optim — PyTorch 2.2 documentationhttps://pytorch.org/docs/stable/optim.html
2.代码:
运行此代码会考验电脑性能,运行速度取决于电脑配置
按照【小土堆】视频中源代码操作,(本人)在执行到第4次时 loss 会开始增长,经过弹幕大佬指正,将优化器 optim 中的 lr 参数调小,可以解决此现象。
同时,(本人)CPU为 i5-12400f 运行20次时间较长,所以将训练次数调制 8 次。
# 优化器
import torch.optim
import torchvision.datasets
from torch import nn
from torch.nn import Conv2d, MaxPool2d, Flatten, Linear, Sequential
from torch.utils.data import DataLoader
dataset = torchvision.datasets.CIFAR10("../data_conv2d", train=False, transform=torchvision.transforms.ToTensor(),
download=True)
dataloader = DataLoader(dataset, batch_size=1)
class Tudui(nn.Module):
def __init__(self):
super(Tudui, self).__init__()
self.model1 = Sequential(
Conv2d(3, 32, 5, padding=2),
MaxPool2d(2),
Conv2d(32, 32, 5, padding=2),
MaxPool2d(2),
Conv2d(32, 64, 5, padding=2),
MaxPool2d(2),
Flatten(),
Linear(1024, 64),
Linear(64, 10)
)
def forward(self, x):
x = self.model1(x)
return x
loss = nn.CrossEntropyLoss() # 公式:Loss(x, class) = -x[class] + log{ ∑exp(x[j]) }
tudui = Tudui()
optim = torch.optim.SGD(tudui.parameters(), lr=0.001) # 定义优化器
for epoch in range(8): # 多次训练
running_loss = 0.0
for data in dataloader:
imgs, targets = data
outputs = tudui(imgs)
result_loss = loss(outputs, targets) # 计算实际输出和目标输出之间的差距
optim.zero_grad() # 节点梯度调零 !!!必须
result_loss.backward() # 得到每行可以调节参数的 梯度
optim.step() # 对每个参数调优
running_loss = running_loss + result_loss
print(running_loss)
"""
定义一个优化器 optim = torch.optim.SGD(tudui.parameters(), lr=0.01)
把优化器对网络中每一个参数的梯度清零 optim.zero_grad()
调用反向传播函数求出每个节点的梯度 result_loss.backward()
对每个参数节点调优 optim.step()
"""