title: pytorch 交叉熵的个人理解
date: 2020-01-19 11:54:06
tags:machine learning
关于pytorch 交叉熵的个人理解
最近,查pytorch的官方文档。里面关于交叉熵部分的公式描述不太理解,查了一些资料,简单的写了下公式。大概理解了一下,不过也不知道正确与否。
官方文档描写如下:
This criterion expects a class index in the range [0, C-1][0,C−1] as the target for each value of a 1D tensor of size minibatch; if ignore_index is specified, this criterion also accepts this class index (this index may not necessarily be in the class range).
The loss can be described as:
loss ( x , c l a s s ) = − log ( exp ( x [ c l a s s ] ) ∑ j exp ( x [ j ] ) ) = − x [ c l a s s ] + log ( ∑ j exp ( x [ j ] ) ) l o s s ( x , c l a s s ) = − l o g ( ∑ j e x p ( x [ j ] ) e x p ( x [ c l a s s ] ) ) = − x [ c l a s s ] + l o g ( j ∑ e x p ( x [ j ] ) ) \text{loss}(x, class) = -\log\left(\frac{\exp(x[class])}{\sum_j \exp(x[j])}\right) \\ = -x[class] + \log\left(\sum_j \exp(x[j])\right)loss(x,class) \\ =−log(∑jexp(x[j])exp(x[class]))=−x[class]+log(j∑exp(x[j])) loss(x,class)=−log(∑j