深度学习中的常用的损失函数(PyTorch)

定义

损失函数量化的预测值和真实值之间的差距，被用来观察所训练的模型是否朝着正确的方向在进行优化。也可以将多个损失函数结合起来计算模型的损失值。

分类问题(Classification)

Cross-Entropy Loss

定义：$L_{CE}=-\frac{1}{n}{\sum }_{i=1}^N{g}_{i}log(p_i)$

其中$g_i$是真实值，$p_i$是第$i$个类别的概率。对于二分类问题来说，$N=2$。

代码：
torch.nn.CrossEntropyLoss（多分类）
torch.nn.BCELoss(二分类)

Dice Loss

定义：$DSC=1-\frac{2{\sum }_i\widehat{y_i}{y}_i+s}{{\sum }_i\widehat{y_i}{\sum }_i{y}_i+s}$
优势：收敛快
代码：

#PyTorch
class DiceLoss(nn.Module):
    def __init__(self, weight=None, size_average=True):
        super(DiceLoss, self).__init__()

    def forward(self, inputs, targets, smooth=1):
        
        #comment out if your model contains a sigmoid or equivalent activation layer
        inputs = F.sigmoid(inputs)       
        
        #flatten label and prediction tensors
        inputs = inputs.view(-1)
        targets = targets.view(-1)
        
        intersection = (inputs * targets).sum()                            
        dice = (2.*intersection + smooth)/(inputs.sum() + targets.sum() + smooth)  
        
        return 1 - dice

Focal Loss

定义：$FL=y\times log(\hat{y})\times (1-\hat{y}{)}^{{\rm Y}}$

在医疗图像分割中，Focal loss 可以提高模型对模糊细胞的分割性能。也可以和Dice loss 结合使用。

代码:

#PyTorch
ALPHA = 0.8
GAMMA = 2

class FocalLoss(nn.Module):
    def __init__(self, weight=None, size_average=True):
        super(FocalLoss, self).__init__()

    def forward(self, inputs, targets, alpha=ALPHA, gamma=GAMMA, smooth=1):
        
        #comment out if your model contains a sigmoid or equivalent activation layer
        inputs = F.sigmoid(inputs)       
        
        #flatten label and prediction tensors
        inputs = inputs.view(-1)
        targets = targets.view(-1)
        
        #first compute binary cross-entropy 
        BCE = F.binary_cross_entropy(inputs, targets, reduction='mean')
        BCE_EXP = torch.exp(-BCE)
        focal_loss = alpha * (1-BCE_EXP)**gamma * BCE
                       
        return focal_loss

Jaccard/Intersection over Union (IoU) Loss

定义：$L_{IoU}=1-\frac{{\sum }_i\widehat{y_i}{y}_i}{{\sum }_i\widehat{y_i}+{\sum }_i{y}_i-{\sum }_i\widehat{y_i}{y}_i}$

IoU loss 常在图像分割中使用，可以反应出预测掩膜图像与实际分割图像的相似性。

代码：

#PyTorch
class IoULoss(nn.Module):
    def __init__(self, weight=None, size_average=True):
        super(IoULoss, self).__init__()

    def forward(self, inputs, targets, smooth=1):
        
        #comment out if your model contains a sigmoid or equivalent activation layer
        inputs = F.sigmoid(inputs)       
        
        #flatten label and prediction tensors
        inputs = inputs.view(-1)
        targets = targets.view(-1)
        
        #intersection is equivalent to True Positive count
        #union is the mutually inclusive area of all labels & predictions 
        intersection = (inputs * targets).sum()
        total = (inputs + targets).sum()
        union = total - intersection 
        
        IoU = (intersection + smooth)/(union + smooth)
                
        return 1 - IoU

回归问题(Regression)

Mean Square Error

定义：$MSE=\frac{{\sum }_i^n(y_i-\widehat{y_i}{)}^2}{n}$

代码：torch.nn.MSELoss

Mean Absolute Error

定义：$MAE=\frac{{\sum }_i^n|y_i-\widehat{y_i}|}{n}$

代码：torch.nn.L1Loss

References

1. https://www.kaggle.com/bigironsphere/loss-function-library-keras-pytorch#Jaccard/Intersection-over-Union-(IoU)-Loss
2. https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html
3. https://pytorch.org/docs/stable/generated/torch.nn.MSELoss.html
4. https://pytorch.org/docs/stable/generated/torch.nn.L1Loss.html