损失函数与反向传播，优化器_为什么loss后向传播,优化器可以前进-CSDN博客

本文链接：https://blog.csdn.net/yiyayayayayayi/article/details/126633131

该博客介绍了损失函数在神经网络中的作用，它用于衡量模型预测输出与实际目标之间的差距，并在反向传播过程中指导权重的更新。示例中展示了使用PyTorch实现的卷积神经网络模型，通过CrossEntropyLoss计算损失，并利用SGD优化器进行训练。在训练过程中，损失逐渐减小，表明模型逐步优化。

摘要由CSDN通过智能技术生成

损失函数与反向传播

计算loss的用处

1. 计算实际输出和目标之间的差距

2. 为我们更新输出提供一定的依据（反向传播），grad

import torchvision.datasets
from torch import nn
from torch.nn import Conv2d, MaxPool2d, Flatten, Linear, Sequential
from torch.utils.data import DataLoader

dataset = torchvision.datasets.CIFAR10("./data", train=False, transform=torchvision.transforms.ToTensor(),
                                       download=True)
dataloader = DataLoader(dataset, batch_size=1)


class Model(nn.Module):
    def __init__(self):
        super(Model, self).__init__()
        self.model1 = Sequential(
            Conv2d(3, 32, 5, padding=2),
            MaxPool2d(2),
            Conv2d(32, 32, 5, padding=2),
            MaxPool2d(2),
            Conv2d(32, 64, 5, padding=2),
            MaxPool2d(2),
            Flatten(),
            Linear(1024, 64),
            Linear(64, 10)
        )

    def forward(self, x):
        x = self.model1(x)
        return x
loss = nn.CrossEntropyLoss()
model = Model()
for data in dataloader:
    imgs, targets = data
    outputs = model(imgs)
    # print(outputs)
    # print(targets)
    result_loss = loss(outputs, targets)
    # print(result_loss)
    result_loss.backward()
    print("ok")

优化器 optimizer

SGD的示例

import torch.optim
import torchvision.datasets
from torch import nn
from torch.nn import Conv2d, MaxPool2d, Flatten, Linear, Sequential
from torch.utils.data import DataLoader

dataset = torchvision.datasets.CIFAR10("./data", train=False, transform=torchvision.transforms.ToTensor(),
                                       download=True)
dataloader = DataLoader(dataset, batch_size=64)


class Model(nn.Module):
    def __init__(self):
        super(Model, self).__init__()
        self.model1 = Sequential(
            Conv2d(3, 32, 5, padding=2),
            MaxPool2d(2),
            Conv2d(32, 32, 5, padding=2),
            MaxPool2d(2),
            Conv2d(32, 64, 5, padding=2),
            MaxPool2d(2),
            Flatten(),
            Linear(1024, 64),
            Linear(64, 10)
        )

    def forward(self, x):
        x = self.model1(x)
        return x
loss = nn.CrossEntropyLoss()
model = Model()
optim  = torch.optim.SGD(model.parameters(), lr=0.01)
for epoch in range(20):
    running_loss = 0.0
    for data in dataloader:
        imgs, targets = data
        outputs = model(imgs)
        # print(outputs)
        # print(targets)
        result_loss = loss(outputs, targets)
        # print(result_loss)
        optim.zero_grad()
        result_loss.backward()
        optim.step()
        running_loss = running_loss + result_loss
    print(running_loss)