loss损失函数

最新推荐文章于 2024-03-08 09:45:00 发布

@天翼:-O

最新推荐文章于 2024-03-08 09:45:00 发布

阅读量276

点赞数

文章标签：深度学习人工智能 pytorch

本文链接：https://blog.csdn.net/qq_48687044/article/details/134595065

版权

1. loss损失函数一方面计算实际输出和目标之间的差距
2. loss损失函数另一方面为我们更新输出提供一定依据
L1和L2都可以做损失函数使用。
1. L2损失函数
L2范数损失函数，也被称为最小平方误差（LSE）。它是把目标值 $eq?y_%7Bi%7D$ 与估计值 f( $eq?x_%7Bi%7D$ )
的差值的平方和最小化。一般回归问题会使用此损失，离群点对次损失影响较大。

$eq?L%20%3D%20%5Csum_%7Bi%20%3D%201%7D%5E%7Bn%7D%28y_%7Bi%7D%20-%20f%28x_%7Bi%7D%29%29%5E%7B2%7D$

2. L1损失函数
也被称为最小绝对值偏差（LAD），绝对值损失函数（LAE）。总的说来，它是把目标值 $eq?y_%7Bi%7D$
与估计值 f( $eq?x_%7Bi%7D$ )的绝对差值的总和最小化。

$eq?L%20%3D%20%5Csum_%7Bi%20%3D%201%7D%5E%7Bn%7D%7Cy_%7Bi%7D%20-%20f%28x_%7Bi%7D%29%7C$

3. 二者对比
L1损失函数相比于L2损失函数的鲁棒性更好。

因为L2范数将误差平方化（如果误差大于1，则误差会放大很多），模型的误差会比L1范数大的多，因此模型会对这种类型的样本更加敏感，这就需要调整模型来最小化误差。但是很大可能这种类型的样本是一个异常值，模型就需要调整以适应这种异常值，那么就导致训练模型的方向偏离目标了。

import torch
from torch.nn import L1Loss
inputs = torch.tensor([1,2,3],dtype=torch.float32)
targets = torch.tensor([1,2,5],dtype=torch.float32)
inputs = torch.reshape(inputs,(1,1,1,3))
targets = torch.reshape(targets,(1,1,1,3))
loss = L1Loss()  # 默认为 maen
result = loss(inputs,targets)
print(result)


import torch
from torch.nn import L1Loss
inputs = torch.tensor([1,2,3],dtype=torch.float32)
targets = torch.tensor([1,2,5],dtype=torch.float32)
inputs = torch.reshape(inputs,(1,1,1,3))
targets = torch.reshape(targets,(1,1,1,3))
loss = L1Loss(reduction='sum') # 修改为sum，三个值的差值，然后取和
result = loss(inputs,targets)
print(result)

3. MSE损失函数

$eq?%5Cvarphi%20%28x%2C%20y%29%20%3D%20L%20%3D%20%5Cleft%20%5C%7Bl%20_%7B1%7D%2C.....l_%7BN%7D%20%5Cright%20%5C%7D%5E%7BT%7D%2C%20l_%7Bn%7D%20%3D%20%28x_%7Bn%7D%20-y%20_%7Bn%7D%29%5E%7B2%7D%2C%20%5Cvarphi%20%28x%2C%20y%29%20%3D%20%5Cbegin%7Bcases%7D%20mean%28L%29%2C%20%26%20%5Ctext%7B%20if%20reduction%7D%20%3D%22mean%22%3B%5C%5C%20sum%28L%29%2C%20%26%20%5Ctext%7B%20if%20reduction%7D%20%3D%22sum%22.%20%5Cend%7Bcases%7D$

import torch
from torch.nn import L1Loss
from torch import nn
inputs = torch.tensor([1,2,3],dtype=torch.float32)
targets = torch.tensor([1,2,5],dtype=torch.float32)
inputs = torch.reshape(inputs,(1,1,1,3))
targets = torch.reshape(targets,(1,1,1,3))
loss_mse = nn.MSELoss()
result_mse = loss_mse(inputs,targets)
print(result_mse)

4.交叉熵损失函数

import torch
from torch.nn import L1Loss
from torch import nn

x = torch.tensor([0.1,0.2,0.3])
y = torch.tensor([1])
x = torch.reshape(x,(1,3)) # 1的 batch_size，有三类
loss_cross = nn.CrossEntropyLoss()
result_cross = loss_cross(x,y)
print(result_cross)

5. 搭建神经网络

import torch
import torchvision
from torch import nn 
from torch.nn import Conv2d, MaxPool2d, Flatten, Linear, Sequential
from torch.utils.data import DataLoader
from torch.utils.tensorboard import SummaryWriter

dataset = torchvision.datasets.CIFAR10("./dataset",train=False,transform=torchvision.transforms.ToTensor(),download=True)       
dataloader = DataLoader(dataset, batch_size=1,drop_last=True)

class Tudui(nn.Module):
    def __init__(self):
        super(Tudui, self).__init__()        
        self.model1 = Sequential(
            Conv2d(3,32,5,padding=2),
            MaxPool2d(2),
            Conv2d(32,32,5,padding=2),
            MaxPool2d(2),
            Conv2d(32,64,5,padding=2),
            MaxPool2d(2),
            Flatten(),
            Linear(1024,64),
            Linear(64,10)
        )
        
    def forward(self, x):
        x = self.model1(x)
        return x
    
tudui = Tudui()
for data in dataloader:
    imgs, targets = data
    outputs = tudui(imgs)
    print(outputs)
    print(targets)

6. 数据集计算损失函数

import torch
import torchvision
from torch import nn 
from torch.nn import Conv2d, MaxPool2d, Flatten, Linear, Sequential
from torch.utils.data import DataLoader
from torch.utils.tensorboard import SummaryWriter

dataset = torchvision.datasets.CIFAR10("./dataset",train=False,transform=torchvision.transforms.ToTensor(),download=True)       
dataloader = DataLoader(dataset, batch_size=64,drop_last=True)

class Tudui(nn.Module):
    def __init__(self):
        super(Tudui, self).__init__()        
        self.model1 = Sequential(
            Conv2d(3,32,5,padding=2),
            MaxPool2d(2),
            Conv2d(32,32,5,padding=2),
            MaxPool2d(2),
            Conv2d(32,64,5,padding=2),
            MaxPool2d(2),
            Flatten(),
            Linear(1024,64),
            Linear(64,10)
        )
        
    def forward(self, x):
        x = self.model1(x)
        return x
    
loss = nn.CrossEntropyLoss() # 交叉熵    
tudui = Tudui()
for data in dataloader:
    imgs, targets = data
    outputs = tudui(imgs)
    result_loss = loss(outputs, targets) # 计算实际输出与目标输出的差距
    print(result_loss)

7. 损失函数反向传播

① 反向传播通过梯度来更新参数，使得loss损失最小；

import torch
import torchvision
from torch import nn 
from torch.nn import Conv2d, MaxPool2d, Flatten, Linear, Sequential
from torch.utils.data import DataLoader
from torch.utils.tensorboard import SummaryWriter

dataset = torchvision.datasets.CIFAR10("./dataset",train=False,transform=torchvision.transforms.ToTensor(),download=True)       
dataloader = DataLoader(dataset, batch_size=64,drop_last=True)

class Tudui(nn.Module):
    def __init__(self):
        super(Tudui, self).__init__()        
        self.model1 = Sequential(
            Conv2d(3,32,5,padding=2),
            MaxPool2d(2),
            Conv2d(32,32,5,padding=2),
            MaxPool2d(2),
            Conv2d(32,64,5,padding=2),
            MaxPool2d(2),
            Flatten(),
            Linear(1024,64),
            Linear(64,10)
        )
        
    def forward(self, x):
        x = self.model1(x)
        return x
    
loss = nn.CrossEntropyLoss() # 交叉熵    
tudui = Tudui()
for data in dataloader:
    imgs, targets = data
    outputs = tudui(imgs)
    result_loss = loss(outputs, targets) # 计算实际输出与目标输出的差距
    result_loss.backward()  # 计算出来的 loss 值有 backward 方法属性，反向传播来计算每个节点的更新的参数。这里查看网络的属性 grad 梯度属性刚开始没有，反向传播计算出来后才有，后面优化器会利用梯度优化网络参数。      
    print("ok")