多类别语义分割中常见 Loss 函数总结

1 综述

学习并整理了一下语义分割的常见Loss，希望能为大家训练语义分割网络的时候提供一些关于Loss方面的知识，之后会不定期更新；【tensorflow实现】

看到一篇2020年论文《A survey of loss functions for semantic segmentation》，文章对目前常见语义分割中Loss functions进行了总结，大家有兴趣可以看看；

论文地址：A survey of loss functions for semantic segmentation
code地址：Semantic-Segmentation-Loss-Functions

1.1 Dice with weight

（1）Dice Loss 对正负样本严重不平衡的场景有着不错的性能，训练过程中更侧重对前景区域的挖掘；
（2）Dice Loss是一种区域相关的loss，像素点的loss以及梯度值不仅和该点的label以及预测值相关，和其他点的label以及预测值也相关；
（3）但训练loss容易不稳定，尤其是小目标的情况下；

Dice with weight代码如下：

def Dice_weight_loss(Y_pred, Y_gt, weight_loss):
    """
    multi label dice loss with weighted
    WDL=1-2*(sum(w*sum(r&p))/sum((w*sum(r+p)))),w=array of shape (C,)
    :param Y_pred: [None, self.image_depth, self.image_height, self.image_width,
                                                       self.numclass],Y_pred is softmax result
    :param Y_gt:[None, self.image_depth, self.image_height, self.image_width,
                                                       self.numclass],Y_gt is one hot result
    :param weight_loss: numpy array of shape (C,) where C is the number of classes
    :return:
    """
    weight_loss = np.array(weight_loss)
    smooth = 1.e-5
    smooth_tf = tf.constant(smooth, tf.float32)
    Y_pred = tf.cast(Y_pred, tf.float32)
    Y_gt = tf.cast(Y_gt, tf.float32)
    # Compute gen dice coef:
    numerator = Y_gt * Y_pred
    numerator = tf.reduce_sum(numerator, axis=(1, 2, 3))
    denominator = Y_gt + Y_pred
    denominator = tf.reduce_sum(denominator, axis=(1, 2, 3))
    gen_dice_coef = tf.reduce_mean(2. * (numerator + smooth_tf) / (denominator + smooth_tf), axis=0)
    loss = -tf.reduce_mean(weight_loss * gen_dice_coef)
    return loss
    

1.2 Tversky loss

论文地址为：Tversky loss function for image segmentation using 3D fully convolutional deep networks

实际上Dice Loss只是Tversky loss的一种特殊形式而已，我们先来看一下Tversky系数的定义，它是Dice系数和Jaccard系数的广义系数。
当 alpha 和 beta 均为0.5的时候，这个公式就是Dice系数，当 alpha 和 beta 均为1的时候，这个公式就是Jaccard系数。

def tversky_loss(Y_pred, Y_gt, beta, weight_loss):
    """
    multi label tversky with weighted
    Tversky loss (TL) is a generalization of Dice loss. TL adds a weight to FP and FN.
    define:TL(p,p')=(p&p')/(p&p'+b*((1-p)&p')+(1-b)*(p&(1-p')))
    :param Y_pred: [None, self.image_depth, self.image_height, self.image_width,
                                                       self.numclass],Y_pred is softmax result
    :param Y_gt:[None, self.image_depth, self.image_height, self.image_width,
                                                       self.numclass],Y_gt is one hot result
    :param beta:beta=1/2,just Dice loss,beta must(0,1)
    :return:
    """
    weight_loss = np.array(weight_loss)
    smooth = 1.e-5
    smooth_tf = tf.constant(smooth, tf.float32)
    Y_pred = tf.cast(Y_pred, tf.float32)
    Y_gt = tf.cast(Y_gt, tf.float32)
    p0 = Y_pred
    p1 = 1 - Y_pred
    g0 = Y_gt
    g1 = 1 - Y_gt
    # Compute gen dice coef:
    numerator = p0 * g0
    numerator = tf.reduce_sum(numerator, axis=(1, 2, 3))
    denominator = tf.reduce_sum(beta * p0 * g1, axis=(1, 2, 3)) + tf.reduce_sum((1 - beta) * p1 * g0, axis=(1, 2, 3)) + numerator
    gen_dice_coef = tf.reduce_mean((numerator + smooth_tf) / (denominator + smooth_tf), axis=0)
    loss = -tf.reduce_mean(weight_loss * gen_dice_coef)
    return loss
    

1.3 Focal loss

论文地址：Focal Loss for Dense Object Detection

何凯明团队在RetinaNet论文中引入了Focal Loss来解决难易样本数量不平衡，gamma用来控制易分样本和难分样本的权重，alpha用来控制正负样本的权重。

Focal loss模块代码如下:

def Focal_loss(Y_pred, Y_gt, gamma, alpha):
    """
     focal_loss between an output and a target
    :param Y_pred: A tensor resulting from a softmax(-1,z,h,w,numclass)
    :param Y_gt: A tensor of the same shape as `y_pred`
    :param alpha: Sample category weight,which is shape (C,) where C is the number of classes
    :param gamma: Difficult sample weight
    :return:
    """
    weight_loss = np.array(alpha)
    epsilon = 1.e-5
    # Scale predictions so that the class probas of each sample sum to 1
    output = Y_pred / tf.reduce_sum(Y_pred, axis=- 1, keepdims=True)
    # Clip the prediction value to prevent NaN's and Inf's
    output = tf.clip_by_value(output, epsilon, 1. - epsilon)
    # Calculate Cross Entropy
    cross_entropy = -Y_gt * tf.log(output)
    # Calculate Focal Loss
    loss = tf.pow(1 - output, gamma) * cross_entropy
    loss = tf.reduce_sum(loss, axis=(1, 2, 3))
    loss = tf.reduce_mean(loss, axis=0)
    loss = tf.reduce_mean(weight_loss * loss)
    return loss

1.4 Generalized dice with weight

论文地址：Generalized Overlap Measures for Evaluation and Validation in Medical Image Analysis

因 Dice Loss 对小目标的预测是十分不利的，因为一旦小目标有部分像素预测错误，就可能会引起Dice系数大幅度波动，导致梯度变化大训练不稳定。另外从上面的代码实现可以发现，Dice Loss针对的是某一个特定类别的分割的损失。当类似于病灶分割有多个场景的时候一般都会使用多个Dice Loss，所以Generalized Dice loss就是将多个类别的Dice Loss进行整合，使用一个指标作为分割结果的量化指标。

Generalized dice with weight代码如下：

def Generalized_dice_loss_w(Y_pred, Y_gt):
    """
    Generalized Dice Loss with class weights
    GDL=1-2*(sum(w*sum(r*p))/sum((w*sum(r+p)))),w=1/sum(r)*sum(r)
    rln为类别l在第n个像素的标准值(GT)，而pln为相应的预测概率值。此处最关键的是wl,为每个类别的权重
    :param Y_gt:[None, self.image_depth, self.image_height, self.image_width,
                                                       self.numclass],Y_gt is one hot result
    :param Y_pred:[None, self.image_depth, self.image_height, self.image_width,
                                                       self.numclass],Y_pred is softmax result
    :return:
    """
    smooth = 1.e-5
    smooth_tf = tf.constant(smooth, tf.float32)
    Y_pred = tf.cast(Y_pred, tf.float32)
    Y_gt = tf.cast(Y_gt, tf.float32)
    # Compute weights: "the contribution of each label is corrected by the inverse of its volume"
    weight_loss = tf.reduce_sum(Y_gt, axis=(0, 1, 2, 3))
    weight_loss = 1 / (tf.pow(weight_loss, 2) + smooth_tf)
    # Compute gen dice coef:
    numerator = Y_gt * Y_pred
    numerator = weight_loss * tf.reduce_sum(numerator, axis=(0, 1, 2, 3))
    numerator = tf.reduce_sum(numerator)
    denominator = Y_gt + Y_pred
    denominator = weight_loss * tf.reduce_sum(denominator, axis=(0, 1, 2, 3))
    denominator = tf.reduce_sum(denominator)
    loss = -2 * (numerator + smooth_tf) / (denominator + smooth_tf)
    return loss

1.5 Cross Entropy with weight

Cross entropy with weight代码如下：

def weighted_categorical_crossentropy(Y_pred, Y_gt, weights):
    """
    weighted_categorical_crossentropy between an output and a target
    loss=-weight*y*log(y')
    :param Y_pred:A tensor resulting from a softmax
    :param Y_gt:A tensor of the same shape as `output`
    :param weights:numpy array of shape (C,) where C is the number of classes
    :return:categorical_crossentropy loss
    Usage:
    weights = np.array([0.5,2,10]) # Class one at 0.5, class 2 twice the normal weights, class 3 10x.
    """
    weights = np.array(weights)
    epsilon = 1.e-5
    # scale preds so that the class probas of each sample sum to 1
    output = Y_pred / tf.reduce_sum(Y_pred, axis=- 1, keep_dims=True)
    # manual computation of crossentropy
    output = tf.clip_by_value(output, epsilon, 1. - epsilon)
    loss = - Y_gt * tf.log(output)
    loss = tf.reduce_sum(loss, axis=(1, 2, 3))
    loss = tf.reduce_mean(loss, axis=0)
    loss = tf.reduce_mean(weights * loss)
    return loss

1.6 Dice + crossEntropy

Dice + crossEntropy代码如下：

def dice_and_crossentroy(Y_pred, Y_gt, weightdice, weightfocal, lamda1=100, lamda2=1):
    """
    hybrid loss function from dice loss and crossentroy
    loss=Ldice+lamda*Lfocalloss
    :param Y_pred:A tensor resulting from a softmax(-1,z,h,w,numclass)
    :param Y_gt: A tensor of the same shape as `y_pred`
    :param gamma:Difficult sample weight
    :param alpha:Sample category weight,which is shape (C,) where C is the number of classes
    :param lamda:trade-off between dice loss and focal loss,can set 0.1,0.5,1
    :return:diceplusfocalloss
    """
    weight_loss1 = np.array(weightdice)
    weight_loss2 = np.array(weightfocal)
    smooth = 1.e-5
    smooth_tf = tf.constant(smooth, tf.float32)
    Y_pred = tf.cast(Y_pred, tf.float32)
    Y_gt = tf.cast(Y_gt, tf.float32)
    # Compute gen dice coef:
    numerator = Y_gt * Y_pred
    numerator = tf.reduce_sum(numerator, axis=(1, 2, 3))
    denominator = Y_gt + Y_pred
    denominator = tf.reduce_sum(denominator, axis=(1, 2, 3))
    gen_dice_coef = tf.reduce_sum(2. * (numerator + smooth_tf) / (denominator + smooth_tf), axis=0)
    loss1 = tf.reduce_mean(weight_loss1 * gen_dice_coef)
    epsilon = 1.e-5
    # scale preds so that the class probas of each sample sum to 1
    output = Y_pred / tf.reduce_sum(Y_pred, axis=- 1, keep_dims=True)
    # manual computation of crossentropy
    output = tf.clip_by_value(output, epsilon, 1. - epsilon)
    loss = -Y_gt * tf.log(output)
    loss = tf.reduce_mean(loss, axis=(1, 2, 3))
    loss = tf.reduce_mean(loss, axis=0)
    loss2 = tf.reduce_mean(weight_loss2 * loss)
    total_loss = lamda1 * (1 - loss1) + lamda2 * loss2
    return total_loss

1.7 Dice + Focal loss

Dice + Focal loss代码如下：

def dice_and_Focalloss(Y_pred, Y_gt, weightdice, weightfocal, lamda1=100, lamda2=1, gamma=4.):
    """
    hybrid loss function from dice loss and focalloss
    loss=Ldice+lamda*Lfocalloss
    :param Y_pred:A tensor resulting from a softmax(-1,z,h,w,numclass)
    :param Y_gt: A tensor of the same shape as `y_pred`
    :param gamma:Difficult sample weight
    :param alpha:Sample category weight,which is shape (C,) where C is the number of classes
    :param lamda:trade-off between dice loss and focal loss,can set 0.1,0.5,1
    :return:dicePfocalloss
    """
    weight_loss1 = np.array(weightdice)
    weight_loss2 = np.array(weightfocal)
    smooth = 1.e-5
    smooth_tf = tf.constant(smooth, tf.float32)
    Y_pred = tf.cast(Y_pred, tf.float32)
    Y_gt = tf.cast(Y_gt, tf.float32)
    # Compute gen dice coef:
    numerator = Y_gt * Y_pred
    numerator = tf.reduce_sum(numerator, axis=(1, 2, 3))
    denominator = Y_gt + Y_pred
    denominator = tf.reduce_sum(denominator, axis=(1, 2, 3))
    gen_dice_coef = tf.reduce_sum(2. * (numerator + smooth_tf) / (denominator + smooth_tf), axis=0)
    loss1 = tf.reduce_mean(weight_loss1 * gen_dice_coef)
    epsilon = 1.e-5
    # Scale predictions so that the class probas of each sample sum to 1
    output = Y_pred / tf.reduce_sum(Y_pred, axis=- 1, keepdims=True)
    # Clip the prediction value to prevent NaN's and Inf's
    output = tf.clip_by_value(output, epsilon, 1. - epsilon)
    # Calculate Cross Entropy
    cross_entropy = -Y_gt * tf.log(output)
    # Calculate Focal Loss
    loss = tf.pow(1 - output, gamma) * cross_entropy
    loss = tf.reduce_mean(loss, axis=(1, 2, 3))
    loss = tf.reduce_mean(loss, axis=0)
    loss2 = tf.reduce_mean(weight_loss2 * loss)
    total_loss = lamda1 * (1 - loss1) + lamda2 * loss2
    return total_loss
    

参考文献

1、【损失函数合集】超详细的语义分割中的Loss大盘点
2、Tensorflow入门教程（四十七）——语义分割损失函数总结