cross entropy loss
简单复习下,在多分类任务中,使用的损失函数通常为交叉熵损失函数。公式如下:
l o s s = ∑ y i t r u e ∗ l o g ( p i ) loss = \sum y_{itrue} * log( p_i) loss=∑yitrue∗log(pi)
使用numpy 跟 torch 实现代码为:
def np_ce(y_pre, y):
return -np.sum( y * np.log(y_pre))
def t_ce(y_pre, y)
return - torch.sum(y * torch.log(y_pre))
torch.CrossEntropyLoss() 坑
为啥说这里坑呢?举个例子:
x.shape >> [m, n]
y.shape >> [m] or [m, labels] (labels expression by one-hot)
在pytorch中,不可以使用两个vector作为api 的输入。 如果自己做出的预测直接是一个vector, 而对应的答案也是一个vector(例如 one-hot) 会报错。pytorch说明文档这样写:
def forward(self, input: Tensor, target: Tensor) -> Tensor:
return F.l1_loss(input, target, reduction=self.reduction)
- Input: (N, C) where C = number of classes, or (N, C, d 1 d_1 d1, d 2 d_2 d2, …, d K d_K dK) with K ≥ 1 K \geq 1 K≥1 in the case of K-dimensional loss.
- Target: (N) where each value is 0 ≤ targets [ i ] ≤ C − 10 ≤ t a r g e t s [ i ] ≤ C − 1 , o r ( N , d 1 , d 2 , . . . , d K ) 0 \leq \text{targets}[i] \leq C-10≤targets[i]≤C−1 , or (N, d_1, d_2, ..., d_K) 0≤targets[i]≤C−10≤targets[i]≤C−1,or(N,d1,d2,...,dK) with K ≥ 1 \geq 1 ≥1 in the case of K-dimensional loss.
- Output: scalar.
解决one-hot 类型为target 输入的方案
得到one-hot 的 index, vector —> scala
- loss_fn(y_pre, torch.max(one_hot_vector, dim = 1))
自己写loss
class cross_en(torch.nn.Module):
def __init__(self):
super(cross_en, self).__init__()
def forward(self ,y_true, y):
loss = - torch.sum(y_true * torch.log(y))
return loss