loss函数之SoftMarginLoss

SoftMarginLoss

用于二分类任务

对于包含$N$个样本的batch数据 $D(x,y)$, $x$代表模型输出，$y$代表真实的类别标签，$y$中元素的值属于 { 1 , − 1 } \{1,-1\} {1,−1}。$loss$计算如下:

$$loss=\frac{{\sum }_i\log (1+\exp (-y[i]\ast x[i]))}{\text{ x.nelement }()}$$

$\text{ x.nelement }()$代表$x$中元素的个数

若每个样本对应一个二分类，则$\text{ x.nelement }()=N$
若每个样本对应于M个二分类，则$\text{ x.nelement }()==M\ast N$

• 当 $x[i]$与$y[i]$同号，即预测正确时，$\exp (-y[i]\ast x[i])<1$, $\log (1+\exp (-y[i]\ast x[i]))<log2$, 值很小。并且$y[i]\ast x[i]$乘积越大，分类确信度越高，loss越小；

• 当 $x[i]$与$y[i]$异号，即预测错误时，$\log (1+\exp (-y[i]\ast x[i]))$取值较大；

$loss$取值$\log (1+\exp (-y[i]\ast x[i]))$而不是$\log (\exp (-y[i]\ast x[i]))$，是为了避免$loss$计算为负数

例子：

import torch
import torch.nn as nn
import math

def validate_SoftMarginLoss(input, target):
    val = 0
    for li_x, li_y in zip(input, target):
        for x, y in zip(li_x, li_y):
            loss_val = math.log(1 + math.exp(- y * x), math.e)
            val += loss_val
    return val / input.nelement()

    
x = torch.FloatTensor([[0.1, 0.2, 0.4, 0.8], [0.1, 0.2, 0.4, 0.8]])
print(x.size())
y = torch.FloatTensor([[1, -1, 1, 1], [1, -1, 1, 1]])
print(y.size())

loss = nn.SoftMarginLoss(reduction="none")
loss_val = loss(x, y)
print(loss_val)

loss = nn.SoftMarginLoss(reduction="sum")
loss_val = loss(x, y)
print(loss_val.item())
print(loss_val.item() / x.nelement())

loss = nn.SoftMarginLoss(reduction="mean")
loss_val = loss(x, y)
print(loss_val.item())

valid_loss_val = validate_SoftMarginLoss(x, y)
print(valid_loss_val)

结果：

torch.Size([2, 4])
torch.Size([2, 4])
tensor([[0.6444, 0.7981, 0.5130, 0.3711],
        [0.6444, 0.7981, 0.5130, 0.3711]])
4.653303146362305
0.5816628932952881
0.5816628932952881
0.5816628606614725