torch.nn.functional.cross_entropy()和torch.nn.CrossEntropyLoss()的使用

最新推荐文章于 2024-07-27 20:20:27 发布

敲代码的小风

最新推荐文章于 2024-07-27 20:20:27 发布

阅读量9.8k

点赞数 11

分类专栏：零基础学习SSD网络PyTorch实现 Deep-Learning-with-PyTorch 《深度学习之PyTorch实战计算机视觉》

本文链接：https://blog.csdn.net/m0_46653437/article/details/111641543

版权

零基础学习SSD网络PyTorch实现同时被 3 个专栏收录

293 篇文章 26 订阅

订阅专栏

Deep-Learning-with-PyTorch

216 篇文章 19 订阅

订阅专栏

《深度学习之PyTorch实战计算机视觉》

198 篇文章 41 订阅

订阅专栏

参考链接: torch.nn.functional.cross_entropy(input, target, weight=None, size_average=None, ignore_index=-100, reduce=None, reduction=‘mean’)
参考链接: class torch.nn.CrossEntropyLoss(weight=None, size_average=None, ignore_index=-100, reduce=None, reduction=‘mean’)

在这里插入图片描述

参数使用简单说明:

weight 
	设置不同类别的权重
size_average 
	已废弃
ignore_index 
	计算损失函数时,忽略某些类别
reduce 
	已经废弃
reduction 
	多种选择'none' | 'mean' | 'sum',指明是否需要求和或者求均值,或者什么都不做.

代码实验展示:

Microsoft Windows [版本 10.0.18363.1256]
(c) 2019 Microsoft Corporation。保留所有权利。

C:\Users\chenxuqi>conda activate ssd4pytorch1_2_0

(ssd4pytorch1_2_0) C:\Users\chenxuqi>python
Python 3.7.7 (default, May  6 2020, 11:45:54) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> from torch.nn import functional as F
>>> from torch import nn as nn
>>> torch.manual_seed(seed=20200910)
<torch._C.Generator object at 0x00000229DDB1D330>
>>>
>>> loss = nn.CrossEntropyLoss()
>>> loss
CrossEntropyLoss()
>>>
>>> input = torch.randn(3, 5, requires_grad=True)
>>> input
tensor([[ 0.2824, -0.3715,  0.9088, -1.7601, -0.1806],
        [ 2.0937,  1.0406, -1.7651,  1.1216,  0.8440],
        [ 0.1783,  0.6859, -1.5942, -0.2006, -0.4050]], requires_grad=True)
>>>
>>> target = torch.empty(3, dtype=torch.long).random_(5)
>>> target
tensor([3, 0, 2])
>>> output = loss(input, target)
>>> output
tensor(2.4510, grad_fn=<NllLossBackward>)
>>> output.backward()
>>>
>>>
>>>
>>>
>>> #------函数形式的使用-------#
>>> input = torch.randn(3, 5, requires_grad=True)
>>> target = torch.randint(5, (3,), dtype=torch.int64)
>>> input
tensor([[-0.5556, -1.5293, -1.8297, -1.8935,  0.4766],
        [-0.9571, -0.9855, -0.9545,  1.0292, -1.7257],
        [ 0.6164,  0.9148, -0.7337,  0.0671,  2.0539]], requires_grad=True)
>>> target
tensor([1, 4, 0])
>>> loss = F.cross_entropy(input, target)
>>> loss
tensor(2.5563, grad_fn=<NllLossBackward>)
>>>
>>>
>>> torch.manual_seed(seed=20200910)
<torch._C.Generator object at 0x00000229DDB1D330>
>>>
>>> input = torch.randn(3, 5, requires_grad=True)
>>> target = torch.randint(5, (3,), dtype=torch.int64)
>>> input
tensor([[ 0.2824, -0.3715,  0.9088, -1.7601, -0.1806],
        [ 2.0937,  1.0406, -1.7651,  1.1216,  0.8440],
        [ 0.1783,  0.6859, -1.5942, -0.2006, -0.4050]], requires_grad=True)
>>> target
tensor([3, 0, 2])
>>> loss = F.cross_entropy(input, target)
>>> loss
tensor(2.4510, grad_fn=<NllLossBackward>)
>>> loss.backward()
>>>
>>>
>>>
>>>

Microsoft Windows [版本 10.0.18363.1256]
(c) 2019 Microsoft Corporation。保留所有权利。

C:\Users\chenxuqi>conda activate ssd4pytorch1_2_0

(ssd4pytorch1_2_0) C:\Users\chenxuqi>python
Python 3.7.7 (default, May  6 2020, 11:45:54) [MSC v.1916 64 bit (AMD64)] :: Anaconda, Inc. on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import torch
>>> from torch.nn import functional as F
>>> torch.manual_seed(seed=20200910)
<torch._C.Generator object at 0x000001E52801D330>
>>>
>>> input = torch.randn(3, 5, requires_grad=True)
>>> target = torch.randint(5, (3,), dtype=torch.int64)
>>> input
tensor([[ 0.2824, -0.3715,  0.9088, -1.7601, -0.1806],
        [ 2.0937,  1.0406, -1.7651,  1.1216,  0.8440],
        [ 0.1783,  0.6859, -1.5942, -0.2006, -0.4050]], requires_grad=True)
>>> target
tensor([3, 0, 2])
>>>
>>>
>>> loss = F.cross_entropy(input, target,reduction='none')
>>> loss
tensor([3.4656, 0.7104, 3.1771], grad_fn=<NllLossBackward>)
>>> (3.4656 + 0.7104 + 3.1771)/3
2.451033333333333
>>>
>>>
>>>

利用NumPy库手动计算一下损失函数:

Python 3.7.4 (tags/v3.7.4:e09359112e, Jul  8 2019, 20:34:20) [MSC v.1916 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license()" for more information.
>>> import numpy as np
>>> 1.7601 + np.log(np.exp(0.2824)+np.exp(-0.3715)+np.exp(0.9088)+np.exp(-1.7601)+np.exp(-0.1806))
3.4656020147895523
>>> 
>>> 
>>> 
>>> x0 = 1.7601 + np.log(np.exp(0.2824)+np.exp(-0.3715)+np.exp(0.9088)+np.exp(-1.7601)+np.exp(-0.1806))
>>> x0
3.4656020147895523
>>> 
>>> x1 = -2.0937 + np.log(np.exp(2.0937)+np.exp(1.0406)+np.exp(-1.7651)+np.exp(1.1216)+np.exp(0.8440))
>>> x1
0.71041054661577
>>> 
>>> x2 = 1.5942 + np.log(np.exp(0.1783)+np.exp(0.6859)+np.exp(-1.5942)+np.exp(-0.2006)+np.exp(-0.4050))
>>> x2
3.1770945844861913
>>> 
>>> (x0 + x1 + x2) / 3.0
2.4510357152971713
>>> 
>>> 
>>>