[torch]create a new criterion(cross entropy)

本文介绍了如何在PyTorch中创建自定义的交叉熵损失函数(CrossEntropyCriterion),包括考虑权重、平均化和处理零值的细节,并提供了CPU和GPU版本的实现步骤及测试代码。

摘要生成于 C知道 ,由 DeepSeek-R1 满血版支持, 前往体验 >

http://blog.csdn.net/u010167269/article/details/51966427

Introduction

CrossentropyCriterion in torch

CrossEntropyCriterion and ClassNLLCriterion only support with hard targets.

For examples, in multi-class classification task,
suppose we have 4 classes: dog, cat, carrot, book.
For one image whose annotation is “cat”, the label will be P=(0,1,0,0) P = ( 0 , 1 , 0 , 0 ) .
And the output of the model is Q=(0.1,0.7,0.01,0.19) Q = ( 0.1 , 0.7 , 0.01 , 0.19 )
CrossEntropyCriterion(torch)
loss=log(Q2) l o s s = − l o g ( Q 2 )
Even if the label is soft P=(0.3,0.5,0.2,0.1) P = ( 0.3 , 0.5 , 0.2 , 0.1 ) ,
CrossEntropyCriterion(torch) won’t use other classes’ label when calculating the loss.
MyCrossEntropyCriterion
In fact, the crossentropy should be loss=ipilog(qi)=p1log(q1)p2log(q2)... l o s s = − ∑ i p i l o g ( q i ) = − p 1 l o g ( q 1 ) − p 2 l o g ( q 2 ) − . . .

One more thing:
DistKLDivCriterion” is the KL divergence(relative entropy),
(pi)log(pi/qi)=[(pi)log(p

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值