PyTorch-Loss Function and BP

目录

1. Loss Function

1.1 L1Loss

1.2 MSELoss

1.3 CrossEntropyLoss

2. 交叉熵与神经网络模型的结合

2.1 反向传播

1. Loss Function

目的： 

a. 计算预测值与真实值之间的差距;

b. 可通过此条件，进行反向传播。

1.1 L1Loss

import torch
from torch.nn import L1Loss

inputs = torch.tensor([1, 2, 3], dtype=torch.float32)
targets = torch.tensor([1, 2, 5], dtype=torch.float32)
inputs = torch.reshape(inputs, (1, 1, 1, 3))  # 1-batch_size,1-channel,1×3
targets = torch.reshape(targets, (1, 1, 1, 3))
loss = L1Loss()
result = loss(inputs, targets)
print(result)  # tensor(0.6667)
loss1 = L1Loss(reduction='sum')
result1 = loss1(inputs, targets)
print(result1)  # tensor(2.)

1.2 MSELoss

import torch
from torch.nn import L1Loss, MSELoss

inputs = torch.tensor([1, 2, 3], dtype=torch.float32)
targets = torch.tensor([1, 2, 5], dtype=torch.float32)
inputs = torch.reshape(inputs, (1, 1, 1, 3))  # 1-batch_size,1-channel,1×3
targets = torch.reshape(targets, (1, 1, 1, 3))
loss_mse = MSELoss()
res = loss_mse(inputs, targets)
print(res)  # tensor(1.3333)

1.3 CrossEntropyLoss

图片来源于：b站up主 我是土堆

​

It is useful when training a classification problem with C classes. 

​

import torch
from torch import nn

x = torch.tensor([0.1, 0.2, 0.3])
y = torch.tensor([1])
x = torch.reshape(x, (1, 3))  # 1-batch_size,3 classes
loss_cross = nn.CrossEntropyLoss()
res = loss_cross(x, y)
print(res)  # tensor(1.1019)

2. 交叉熵与神经网络模型的结合

nn_loss_network.py

import torchvision
from torch import nn
from torch.nn import Sequential, Conv2d, MaxPool2d, Flatten, Linear
from torch.utils.data import DataLoader

dataset = torchvision.datasets.CIFAR10('./dataset', train=False, transform=torchvision.transforms.ToTensor(),
                                       download=True)
dataloader = DataLoader(dataset, batch_size=1)

class MyModule(nn.Module):
    def __init__(self):
        super(MyModule, self).__init__()
        self.model1 = Sequential(
            Conv2d(3, 32, 5, padding=2),
            MaxPool2d(2),
            Conv2d(32, 32, 5, padding=2),
            MaxPool2d(2),
            Conv2d(32, 64, 5, padding=2),
            MaxPool2d(2),
            Flatten(),
            Linear(1024, 64),
            Linear(64, 10)
        )

    def forward(self, x):
        x = self.model1(x)
        return x

myModule1 = MyModule()
for data in dataloader:
    imgs, targets = data
    outputs = myModule1(imgs)
    print(outputs)
    print(targets)

tensor([[-0.1187,  0.1490, -0.1015,  0.0767, -0.0677, -0.0625,  0.0553, -0.0932,
         -0.0866,  0.0746]], grad_fn=<AddmmBackward0>)
tensor([1])

计算交叉熵损失

loss = nn.CrossEntropyLoss()
myModule1 = MyModule()
for data in dataloader:
    imgs, targets = data
    outputs = myModule1(imgs)
    res_loss = loss(outputs, targets)
    print(res_loss)

tensor(2.4315, grad_fn=<NllLossBackward0>)
tensor(2.3594, grad_fn=<NllLossBackward0>)
tensor(2.3659, grad_fn=<NllLossBackward0>)

...

2.1 反向传播

for data in dataloader:
    imgs, targets = data
    outputs = myModule1(imgs)
    res_loss = loss(outputs, targets)
    res_loss.backward()