pytorch 常用的 loss function

1 nn.L1Loss

$$loss(X_i,y_i)=|X_i-y_i|$$
这里我们亲自做一下实验看看具体效果

# torch.nn.L1Loss

import torch

l1_loss_fn = torch.nn.L1Loss(reduce=False, size_average=False)
input = torch.autograd.Variable(torch.randn(3,4))
target = torch.autograd.Variable(torch.randn(3,4))
loss = l1_loss_fn(input, target)
print(input)
print(target)
print(loss)

res = torch.abs(input-target)
print("loss computed by ourself")
print(res)

结果如下：

tensor([[ 0.5152, -1.3686,  0.3119, -0.3094],
        [-0.3865, -0.2515, -1.4992, -0.2219],
        [ 0.3324, -0.3495,  0.8597, -0.0018]])
tensor([[ 1.3572, -0.9364,  1.0528,  0.4357],
        [-0.2460,  0.2986, -0.5723, -0.1117],
        [-1.1078,  1.1902,  1.4491, -0.2142]])
tensor([[0.8420, 0.4322, 0.7408, 0.7452],
        [0.1405, 0.5502, 0.9268, 0.1102],
        [1.4402, 1.5397, 0.5894, 0.2124]])
tensor([[0.8420, 0.4322, 0.7408, 0.7452],
        [0.1405, 0.5502, 0.9268, 0.1102],
        [1.4402, 1.5397, 0.5894, 0.2124]])
tensor(8.2697)

另外，reduce参数和size_average参数有不同的搭配组合。

1. 当reduce=False时，忽略size_average参数，此时输出的结果的维度与输入一致
2. 当reduce=True，size_average=True时，此时输出的结果=torch.mean(torch.abs(input-target))
3. 当reduce=True，size_average=False时，此时的输出结果=`torch.sum(torch.abs(input-target))

其他Loss function也具有reduce和size_average这两个参数，且作用类似

2 nn.MSELoss()

$$loss(X_i,y_i)=(X_i-y_i{)}^2$$

# torch.nn.MSELoss

import torch

MSE_loss_fn = torch.nn.MSELoss(reduce=False, size_average=False)
input = torch.autograd.Variable(torch.randn(3,4))
target = torch.autograd.Variable(torch.randn(3,4))
loss = MSE_loss_fn(input, target)
print(input)
print(target)
print(loss)

res=input-target
print(res*res)

结果如下：

tensor([[ 0.3487,  0.4603, -0.3404, -0.2632],
        [ 0.5376, -1.0239, -1.5926, -1.2581],
        [ 0.8796,  0.4397, -0.2821,  0.0028]])
tensor([[ 0.9949,  2.3588,  0.1053, -1.2758],
        [-0.5526, -1.0309,  0.9014, -0.0308],
        [ 0.9400,  1.1123,  0.3666, -0.5454]])
tensor([[4.1764e-01, 3.6046e+00, 1.9869e-01, 1.0253e+00],
        [1.1884e+00, 4.8598e-05, 6.2203e+00, 1.5061e+00],
        [3.6437e-03, 4.5234e-01, 4.2089e-01, 3.0058e-01]])
tensor([[4.1764e-01, 3.6046e+00, 1.9869e-01, 1.0253e+00],
        [1.1884e+00, 4.8598e-05, 6.2203e+00, 1.5061e+00],
        [3.6437e-03, 4.5234e-01, 4.2089e-01, 3.0058e-01]])

3 nn.BCELoss()

BCELoss是二分类使用的交叉熵，用之前需要在该层前面加上Sigmoid函数。
$$loss(X_i,y_i)=-w_i[y_{i}log{x}_i+（1-y_i)log(1-x_i)]$$

import torch
import torch.nn.functional as F

BCE_loss_fn = torch.nn.BCELoss(reduce=False, size_average=False)
BCE_logid_loss = torch.nn.BCEWithLogitsLoss(reduce=False, size_average=False)

input=torch.autograd.Variable(torch.randn(3,4))
target=torch.autograd.Variable(torch.FloatTensor(3,4).random_(2))
loss = BCE_loss_fn(F.sigmoid(input), target)
print(input)
print(target)
print(loss)
print(BCE_logid_loss(input,target))

结果如下：

tensor([[-0.2960, -0.6593,  0.7279, -1.1125],
        [ 0.9475,  0.5286,  1.6567, -0.2942],
        [-0.0741,  2.1198,  0.9491,  0.7699]])
tensor([[1., 1., 1., 1.],
        [1., 0., 0., 0.],
        [1., 0., 1., 1.]])
tensor([[0.8521, 1.0762, 0.3940, 1.3967],
        [0.3277, 0.9920, 1.8313, 0.5568],
        [0.7309, 2.2332, 0.3272, 0.3805]])
tensor([[0.8521, 1.0762, 0.3940, 1.3967],
        [0.3277, 0.9920, 1.8313, 0.5568],
        [0.7309, 2.2332, 0.3272, 0.3805]])

可以看出，loss, x, y, w 的维度都是一样的。
此外，使用nn.BCEWithLogitsLoss是不需要使用Sigmoid层

4 nn.CrossEntropyLoss

该函数用于多分类，不需要加softmax层
$$loss(x,label)=-w_{label}log\frac{e^{x_{label}}}{{\sum }_{j=1}^N{e}^{x_j}}$$

import torch

loss_fn = torch.nn.CrossEntropyLoss(reduce=False, size_average=False)
input=torch.autograd.Variable(torch.randn(3,4))
target=torch.autograd.Variable(torch.LongTensor(3).random_(4))
loss = loss_fn(input, target)

print(input)
print(target)
print(loss)

结果如下：

tensor([[-0.2541,  0.5136,  1.2984, -0.1278],
        [ 1.4406,  2.6949,  1.9780,  1.8310],
        [-0.1522,  1.7501, -1.0701, -0.3558]])
tensor([1, 3, 3])
tensor([1.4309, 1.6501, 2.3915])

参考

参考