手动分开计算交叉熵每个位置的loss值

交叉熵loss函数

交叉熵损失函数最后得到的是所有位置的平均loss，如果我们想要知道每个位置的loss多少，可以分开计算，如下例子所示:
import torch

fn = torch.nn.CrossEntropyLoss()

logits = torch.tensor([[0.7595, 0.3526, 0.0479, 0.6684],
        [0.5149, 0.4248, 0.5377, 0.9349],
        [0.0801, 0.2237, 0.2990, 0.3178],
        [0.0583, 0.7461, 0.3659, 0.8972],
        [0.7178, 0.1821, 0.9660, 0.1914],
        [0.5666, 0.8041, 0.9830, 0.0220]])

print(logits)
labels = torch.tensor([[1, 0], [1, 0], [0, 1]]).view(-1)
print(labels)
loss = fn(logits, labels)
print(f'损失函数计算的loss {loss}')
loss_2 = -torch.nn.functional.log_softmax(logits, dim=-1).gather(1, labels.unsqueeze(-1)).squeeze(1)
print(f'每个位置的loss {loss_2}')  # 每个位置的loss
print(f'平均loss {loss_2.mean()}')

结果：

tensor([[0.7595, 0.3526, 0.0479, 0.6684],
        [0.5149, 0.4248, 0.5377, 0.9349],
        [0.0801, 0.2237, 0.2990, 0.3178],
        [0.0583, 0.7461, 0.3659, 0.8972],
        [0.7178, 0.1821, 0.9660, 0.1914],
        [0.5666, 0.8041, 0.9830, 0.0220]])
tensor([1, 0, 1, 0, 0, 1])
损失函数计算的loss 1.4655965566635132
每个位置的loss tensor([1.5284, 1.4949, 1.3970, 1.8966, 1.2406, 1.2360])
平均loss 1.4655965566635132