BECLoss和CELoss代码验证、多分类问题两者loss比较

BCELoss

官网说明：BCELoss — PyTorch 2.0 documentation

m = nn.Sigmoid()
loss = nn.BCELoss(reduction='none')
input = torch.randn(3, requires_grad=True)
target = torch.empty(3).random_(2)
sigInput = m(input)
output = loss(sigInput, target)
print(input)
print(target)
print(output)

for i in range(3):
    label = target[i]
    pre = sigInput[i]
    L = -(label * torch.log(pre) + (1-label) * torch.log(1-pre))
    print(L)

运行结果： 

tensor([ 0.6806,  0.5188, -0.7682], requires_grad=True)
tensor([1., 0., 0.])
tensor([0.4097, 0.9858, 0.3811], grad_fn=<BinaryCrossEntropyBackward>)
tensor(0.4097, grad_fn=<NegBackward>)
tensor(0.9858, grad_fn=<NegBackward>)
tensor(0.3811, grad_fn=<NegBackward>)

其中标签不一定非要为0和1，看YOLOX代码中发现的。即：

target = torch.empty(3).random_(2)*0.9

YOLOX中代码：

cls_target  = F.one_hot(gt_matched_classes.to(torch.int64), self.num_classes).float()
 * pred_ious_this_matching.unsqueeze(-1)

计算loss的标签one-hot编码后乘了iou值。

其实从公式计算的角度，标签是什么都不影响结果的计算，只是算出的结果有没有意义罢了。

CELoss

loss = nn.CrossEntropyLoss(reduction='none')
input = torch.randn(3, 5, requires_grad=True)
target = torch.empty(3, dtype=torch.long).random_(5)
output = loss(input, target)
print(input)
print(target)
print(output)

for i in range(3):
    label = target[i]
    softmaxInput = input.softmax(dim=1)
    pre = softmaxInput[i]
    j = label.numpy()
    lossValue = -torch.log(pre[j])
    print(lossValue)

注意官方文档[CrossEntropyLoss — PyTorch 2.0 documentation]中的计算公式，就是相当于先求softmax在计算。

运行结果： 

tensor([[-0.2283,  1.6527, -0.8229, -1.5192,  0.4447],
        [-0.0529, -0.8231, -0.9425, -1.3467, -0.7180],
        [-0.4223,  0.5041,  1.5112, -0.6260,  0.4895]], requires_grad=True)
tensor([4, 4, 3])
tensor([1.6637, 1.6442, 2.8243], grad_fn=<NllLossBackward>)
tensor(1.6637, grad_fn=<NegBackward>)
tensor(1.6442, grad_fn=<NegBackward>)
tensor(2.8243, grad_fn=<NegBackward>)

多分类问题时用BCEloss和CEloss结果比较

input = torch.tensor([
    [1, 0.2, 0.2, 0.2, 0.2],
    [0.5, 1, 0.5, 0.5, 0.5],
    [1, 0.2, 0.2, 0.2, 0.2]
])
target = torch.tensor([0, 1, 4])
print(input)
print(target)

CEloss = nn.CrossEntropyLoss(reduction='none')
CEOutput = CEloss(input, target)
print(CEOutput)

BECloss = nn.BCELoss(reduction='none')
BECOutput = BECloss(input.softmax(dim=1), F.one_hot(target.to(torch.int64), 5).float())
print(BECOutput.sum(dim=1))

运行结果：

tensor([[1.0000, 0.2000, 0.2000, 0.2000, 0.2000],
        [0.5000, 1.0000, 0.5000, 0.5000, 0.5000],
        [1.0000, 0.2000, 0.2000, 0.2000, 0.2000]])
tensor([0, 1, 4])
tensor([1.0287, 1.2314, 1.8287])
tensor([1.7291, 2.0108, 2.7963])

有问题请指正。