Loss——Focal Loss

Loss——Focal Loss

一、简介

• Focal Loss论文地址：https://arxiv.org/pdf/1708.02002.pdf
• Focal Loss是基于Cross Entropy修改的，是解决分类不均衡问题的一种方案。 reduces the relative loss for well-classified examples，(pt > :5), putting more focus on hard, misclassified examples,加入参数：$\alpha 、\gamma$，这两个参数自己设定。

二、原理、公式推导

$$p_i=sigmoid(x_i)=\frac{1}{1+e^{-x_i}}(1)$$
$$p_t=\{\begin{array}{rlr}{p}_i& & if,y_i=1\\ 1-p_i& & otherwise\end{array}(2)$$
$$CE(p_t)=-log(p_t)=-(y\ast log(p_i)+(1-y)log(1-p_i))(3)$$
$$FL(p_t)=\alpha (1-p_t{)}^{\gamma }\times CE(p_t)=-\alpha (1-p_t{)}^{\gamma }\times log(p_t)=-(\alpha (1-p_i{)}^{\gamma }y\ast log(p_i)+\alpha p_i^{\gamma }(1-y)log(1-p_i))(4)$$

• 反向传播,求梯度, y ∈ { 1 , − 1 } y\in\{1,-1\} y∈{1,−1}。
  $$\frac{\partial FL}{\partial x_i}=\frac{\partial FL}{\partial p_t}\times \frac{\partial p_t}{\partial x_i}(5)$$
  $$\frac{\partial FL}{\partial p_t}=-[-\alpha \gamma (1-p_t{)}^{\gamma -1}log(p_t)+\frac{(1-p_t{)}^{\gamma }}{p_t}](6)$$
  $$\frac{\partial p_t}{\partial x_i}=\frac{\partial p_t}{\partial p_i}\times \frac{\partial p_i}{\partial x_i}=y\times (1-p_i)\times p_i=y\times p_t\times (1-p_t)(7)$$
  $$\frac{\partial FL}{\partial x_i}=\frac{\partial FL}{\partial p_t}\times \frac{\partial p_t}{\partial x_i}=y\alpha (1-p_t{)}^{\gamma }[\gamma p_{t}log(p_t)+p_t-1](8)$$

三、darknet yolov3-spp focal loss 代码（自己添加）

 if (focal_loss) {
        // Focal Loss
        float alpha = 0.5;    // 0.25 or 0.5
        int gamma = 2;
        if (delta[index + stride*class_id]){
            int index_classes = index + stride*class_id;
            float pt = output[index_classes];
            pt = max(pt,  0.000000000000001F);
            float grad = pow(1-pt, gamma) * (gamma * pt * logf(pt) + pt - 1);
            delta[index + stride*class_id] = (-1 * alpha*grad);
            if(avg_cat) *avg_cat += output[index + stride*class_id];
            return;
        }
        for (n = 0; n < classes; ++n) {
            int index_classes = index + stride*n;
            float pt = (n == class_id ? output[index_classes]: (1- output[index_classes]));
            pt = max(pt,  0.000000000000001F);
            float grad = (n == class_id ? pow(1-pt, gamma) * (gamma * pt * logf(pt) + pt - 1): \
                                         -1 * pow(1-pt, gamma) * (gamma * pt * logf(pt) + pt - 1));
            delta[index_classes]  = (-1 * alpha*grad);

            if (n == class_id) *avg_cat += output[index_classes];
        }
    }

四、总结

• paper 中指出 $\gamma =2$时效果最佳。Focal Loss在实际应用中并非都能提升分类能力。