[torch]create a new criterion(cross entropy)

最新推荐文章于 2023-04-02 16:05:50 发布

三枚目

最新推荐文章于 2023-04-02 16:05:50 发布

阅读量3.8k

点赞数

分类专栏： torch

本文链接：https://blog.csdn.net/apsvvfb/article/details/71429695

版权

本文介绍了如何在PyTorch中创建自定义的交叉熵损失函数（CrossEntropyCriterion），包括考虑权重、平均化和处理零值的细节，并提供了CPU和GPU版本的实现步骤及测试代码。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

http://blog.csdn.net/u010167269/article/details/51966427

Introduction

CrossentropyCriterion in torch

CrossEntropyCriterion and ClassNLLCriterion only support with hard targets.

For examples, in multi-class classification task,
suppose we have 4 classes: dog, cat, carrot, book.
For one image whose annotation is “cat”, the label will be $P=(0,1,0,0)$ .
And the output of the model is $Q=(0.1,0.7,0.01,0.19)$
CrossEntropyCriterion(torch)
$loss=-log(Q_2)$
Even if the label is soft $P=(0.3,0.5,0.2,0.1)$ ,
CrossEntropyCriterion(torch) won’t use other classes’ label when calculating the loss.
MyCrossEntropyCriterion
In fact, the crossentropy should be $loss=-\sum_{i}{p_ilog(q_i)}=-p_1log(q_1)-p_2log(q_2)-...$