目录
定义
损失函数量化的预测值和真实值之间的差距,被用来观察所训练的模型是否朝着正确的方向在进行优化。也可以将多个损失函数结合起来计算模型的损失值。
分类问题(Classification)
Cross-Entropy Loss
定义: L C E = − 1 n ∑ i = 1 N g i l o g ( p i ) L_{CE}=-\frac{1}{n}\sum_{i=1}^{N}{g_ilog(p_i)} LCE=−n1∑i=1Ngilog(pi)
其中 g i g_i gi是真实值, p i p_i pi是第 i i i个类别的概率。对于二分类问题来说, N = 2 N=2 N=2。
代码:
torch.nn.CrossEntropyLoss
(多分类)
torch.nn.BCELoss
(二分类)
Dice Loss
定义:
D
S
C
=
1
−
2
∑
i
y
i
^
y
i
+
s
∑
i
y
i
^
∑
i
y
i
+
s
DSC=1-\frac{2\sum_{i}{\hat{y_i}y_i+s}}{\sum_{i}{\hat{y_i}}\sum_{i}y_i+s}
DSC=1−∑iyi^∑iyi+s2∑iyi^yi+s
优势:收敛快
代码:
#PyTorch
class DiceLoss(nn.Module):
def __init__(self, weight=None, size_average=True):
super(DiceLoss, self).__init__()
def forward(self, inputs, targets, smooth=1):
#comment out if your model contains a sigmoid or equivalent activation layer
inputs = F.sigmoid(inputs)
#flatten label and prediction tensors
inputs = inputs.view(-1)
targets = targets.view(-1)
intersection = (inputs * targets).sum()
dice = (2.*intersection + smooth)/(inputs.sum() + targets.sum() + smooth)
return 1 - dice
Focal Loss
定义: F L = y × l o g ( y ^ ) × ( 1 − y ^ ) Υ FL=y\times log(\hat{y})\times (1-\hat{y})^\Upsilon FL=y×log(y^)×(1−y^)Υ
在医疗图像分割中,Focal loss 可以提高模型对模糊细胞的分割性能。也可以和Dice loss 结合使用。
代码:
#PyTorch
ALPHA = 0.8
GAMMA = 2
class FocalLoss(nn.Module):
def __init__(self, weight=None, size_average=True):
super(FocalLoss, self).__init__()
def forward(self, inputs, targets, alpha=ALPHA, gamma=GAMMA, smooth=1):
#comment out if your model contains a sigmoid or equivalent activation layer
inputs = F.sigmoid(inputs)
#flatten label and prediction tensors
inputs = inputs.view(-1)
targets = targets.view(-1)
#first compute binary cross-entropy
BCE = F.binary_cross_entropy(inputs, targets, reduction='mean')
BCE_EXP = torch.exp(-BCE)
focal_loss = alpha * (1-BCE_EXP)**gamma * BCE
return focal_loss
Jaccard/Intersection over Union (IoU) Loss
定义: L I o U = 1 − ∑ i y i ^ y i ∑ i y i ^ + ∑ i y i − ∑ i y i ^ y i L_{IoU}=1-\frac{\sum_{i}{\hat{y_i}y_i}}{\sum_{i}{\hat{y_i}}+\sum_{i}y_i-\sum_{i}{\hat{y_i}y_i}} LIoU=1−∑iyi^+∑iyi−∑iyi^yi∑iyi^yi
IoU loss 常在图像分割中使用,可以反应出预测掩膜图像与实际分割图像的相似性。
代码:
#PyTorch
class IoULoss(nn.Module):
def __init__(self, weight=None, size_average=True):
super(IoULoss, self).__init__()
def forward(self, inputs, targets, smooth=1):
#comment out if your model contains a sigmoid or equivalent activation layer
inputs = F.sigmoid(inputs)
#flatten label and prediction tensors
inputs = inputs.view(-1)
targets = targets.view(-1)
#intersection is equivalent to True Positive count
#union is the mutually inclusive area of all labels & predictions
intersection = (inputs * targets).sum()
total = (inputs + targets).sum()
union = total - intersection
IoU = (intersection + smooth)/(union + smooth)
return 1 - IoU
回归问题(Regression)
Mean Square Error
定义: M S E = ∑ i n ( y i − y i ^ ) 2 n MSE=\frac{\sum_{i}^{n}{(y_i-\hat{y_i})^2}}{n} MSE=n∑in(yi−yi^)2
代码:torch.nn.MSELoss
Mean Absolute Error
定义: M A E = ∑ i n ∣ y i − y i ^ ∣ n MAE=\frac{\sum_{i}^{n}{|y_i-\hat{y_i}|}}{n} MAE=n∑in∣yi−yi^∣
代码:torch.nn.L1Loss
References
- https://www.kaggle.com/bigironsphere/loss-function-library-keras-pytorch#Jaccard/Intersection-over-Union-(IoU)-Loss
- https://pytorch.org/docs/stable/generated/torch.nn.CrossEntropyLoss.html
- https://pytorch.org/docs/stable/generated/torch.nn.MSELoss.html
- https://pytorch.org/docs/stable/generated/torch.nn.L1Loss.html