pytorch交叉熵损失
BCELoss vs BCEWithLogitsLoss
bceloss前面需要经过sigmoid
BCEWithLogitsLoss就是把Sigmoid-BCELoss合成一步
input = torch.tensor(np.arange(3)/3).reshape(3,1)
input2 = torch.nn.Sigmoid()(input)
loss = torch.nn.BCELoss()
target = torch.tensor([[0],[1],[1]]).to(torch.double)
print(loss(input2,target))
loss = torch.nn.BCEWithLogitsLoss()
print (loss(input,target))
#输出
#tensor(0.5493, dtype=torch.float64)
#tensor(0.5493, dtype=torch.float64)
nll_loss和cross_entropy loss
和上面一样,CrossEntropyLoss就是把以上Softmax–Log–NLLLoss合并成一步,注意这里还需要多经过一层log
input = torch.randn(3,3)
input2 = torch.log(torch.nn.Softmax(dim=1)(input))
loss = torch.nn.NLLLoss()
target = torch.tensor([0,2,1])
print(loss(input2,target))
loss = torch.nn.CrossEntropyLoss()
print (loss(input,target))
#输出
#tensor(1.0774)
#tensor(1.0774)
tensorflow/keras交叉熵损失
这里就只讨论多分类的
sparse_categorical_crossentropy 与 sparse_softmax_cross_entropy_with_logits
这个函数对应pytorch的cross_entropy
在tensorflow里的"logits"指的其实是,该方法是在logit数值上使用softmax或者sigmoid来进行normalization的,也暗示用户不要将网络输出进行sigmoid或者softmax,这些过程可以在函数内部更高效地计算。
在torch里:
outputs=torch.tensor([[[1.4,1.2],[0.0,0.0],[0.3,1.8],[0.3,0.0]],[[1.4,1.2],[0.0,0.0],[0.3,1.8],[0.3,0.0]]]).to(torch.float32)
labels =torch.tensor([[0,0],[0,0]])
loss = torch.nn.CrossEntropyLoss()(outputs, labels)
loss,outputs.shape,labels.shape
#输出
#(tensor(0.9396), torch.Size([2, 4, 2]), torch.Size([2, 2]))
在tensorflow/keras里,tf里的sparse_softmax_cross_entropy_wi