深入浅出理解CrossEntropy

import torch
import numpy as np
import torch.nn.functional as F
x = np.array([[1, 2,3,4,5],
             [1, 2,3,4,5],
             [1, 2,3,4,5]]).astype(np.float32)
y = np.array([1, 1, 0])
x = torch.from_numpy(x)
y = torch.from_numpy(y).long()  #标签为long型
soft_out = F.softmax(x,dim=1)
log_soft_out1 = torch.log(soft_out)
print("log_soft_out1:{}".format(log_soft_out1))
log_soft_out2 = F.log_softmax(soft_out)
print("log_soft_out2:{}".format(log_soft_out1))
loss = F.nll_loss(log_soft_out1, y)
print(soft_out)
print(log_soft_out1)
print(loss)
loss = F.cross_entropy(x, y)
print(loss)

结果：

log_soft_out1:tensor([[-4.4519, -3.4519, -2.4519, -1.4519, -0.4519],
        [-4.4519, -3.4519, -2.4519, -1.4519, -0.4519],
        [-4.4519, -3.4519, -2.4519, -1.4519, -0.4519]])
log_soft_out2:tensor([[-4.4519, -3.4519, -2.4519, -1.4519, -0.4519],
        [-4.4519, -3.4519, -2.4519, -1.4519, -0.4519],
        [-4.4519, -3.4519, -2.4519, -1.4519, -0.4519]])
tensor([[0.0117, 0.0317, 0.0861, 0.2341, 0.6364],
        [0.0117, 0.0317, 0.0861, 0.2341, 0.6364],
        [0.0117, 0.0317, 0.0861, 0.2341, 0.6364]])
tensor([[-4.4519, -3.4519, -2.4519, -1.4519, -0.4519],
        [-4.4519, -3.4519, -2.4519, -1.4519, -0.4519],
        [-4.4519, -3.4519, -2.4519, -1.4519, -0.4519]])
tensor(3.7852)
tensor(3.7852)

结果分析：

F.softmax(x,dim=1)：一行和为1 sum([0.0117, 0.0317, 0.0861, 0.2341, 0.6364])=1
softmax函数公式

torch.log(soft_out)：对softmax的结果进行取对数

F.nll_loss(log_soft_out, y)：对取对数的结果，根据y的值，（y值是索引），找到对应的值，黄色部分，各自取相反数再相加，求平均
（3.4519+3.4519+4.4519）/3 = 3.7852
所以：
cross_entropy函数：softmax->log->nll_loss