# 交叉熵损失(Cross Entropy)求导

Cross Entropy是分类问题中常见的一种损失函数，我们在之前的文章提到过二值交叉熵的证明交叉熵的作用，下面解释一下交叉熵损失的求导。

$p_{i} = \frac{e^{f_{i}}}{\sum_{k=0}^{C-1} e^{f_{k}}}$

$L = -\sum_{i=0}^{C-1} y_{i}log^{p_{i}}$

$\frac{\partial L}{\partial f_{i}} = \sum_{j=0}^{C-1} \frac{\partial L_{j}}{\partial p_{j}}\frac{\partial p_{j}}{\partial f_{i}}$

$\frac{\partial L_{j}}{\partial p_{j}}= \frac{\partial (-y_{j}log^{p_{j}})}{\partial (p_{j})}$

$\frac{\partial L_{j}}{\partial p_{j}}= \frac{\partial (-y_{j}log^{p_{j}})}{\partial (p_{j})} =-\frac{y_{j}}{p_{j}}$

• 首先$j=i$时：
$\frac{\partial p_{j}}{\partial f_{i}} = \frac{\partial p_{i}}{\partial f_{i}} = \frac{\partial \frac{e^{f_{i}}}{\sum_{k=0}^{C-1} e^{f_{k}}}}{\partial f_{i}}$
$= \frac{ (e^{f_{i}})' \sum_{k=0}^{C-1} e^{f_{k}} - e^{f_{i}}(\sum_{k=0}^{C-1} e^{f_{k}})' }{(\sum_{k=0}^{C-1} e^{f_{k}})^{2}}$
$= \frac{ e^{f_{i}}\sum_{k=0}^{C-1} e^{f_{k}} - (e^{f_{i}})^2 }{(\sum_{k=0}^{C-1} e^{f_{k}})^{2}}= \frac{ e^{f_{i}} }{\sum_{k=0}^{C-1} e^{f_{k}}} - (\frac{ e^{f_{i}} }{\sum_{k=0}^{C-1} e^{f_{k}}})^2$
$= p_{i}-(p{i})^2 = p_{i}(1-p_{i})$

• 然后$j\neq i$时：
$\frac{\partial p_{j}}{\partial f_{i}}= \frac{\partial \frac{e^{f_{j}}}{\sum_{k=0}^{C-1} e^{f_{k}}}}{\partial f_{i}}$
$= \frac{ (e^{f_{j}})' \sum_{k=0}^{C-1} e^{f_{k}} - e^{f_{j}}(\sum_{k=0}^{C-1} e^{f_{k}})' }{(\sum_{k=0}^{C-1} e^{f_{k}})^{2}}$
$= \frac{ - e^{f_{i}} e^{f_{j}} }{(\sum_{k=0}^{C-1} e^{f_{k}})^{2}} = - \frac{ e^{f_{i}} }{\sum_{k=0}^{C-1} e^{f_{k}}} \frac{ e^{f_{j}} }{\sum_{k=0}^{C-1} e^{f_{k}}}$
$= -p_{i}p_{j}$

$\frac{\partial L}{\partial f_{i}} = \sum_{j=i}^{C-1} \frac{\partial L_{j}}{\partial p_{j}}\frac{\partial p_{j}}{\partial f_{i}} + \sum_{j\neq i}^{C-1} \frac{\partial L_{j}}{\partial p_{j}}\frac{\partial p_{j}}{\partial f_{i}}$
$=-\frac{y_{i}}{p_{i}}p_{i}(1-p_{i}) + \sum_{j\neq i}^{C-1}-p_{i}p_{j}(-\frac{y_{j}}{p_{j}})$
$=-y_{i}(1-p_{i}) + \sum_{j\neq i}^{C-1}p_{i}y_{j}$
$=y_{i}p_{i}-y_{i} + \sum_{j\neq i}^{C-1}p_{i}y_{j}$

$=\sum_{j=0}^{C-1}p_{i}y_{j} - y_{i}$
$=p_{i}\sum_{j=0}^{C-1}y_{j} - y_{i}$
$\sum_{j=0}^{C-1}y_{j} = 1$，所以：
$=p_{i}\sum_{j=0}^{C-1}y_{j} - y_{i} = p_{i}-y_{i}$

08-27 7120

11-12 665

07-01 782

07-25 1394

07-26 6794

10-16 8451

09-25 1万+

01-02 2万+

06-29 557

08-19 1987

#### softmax激活+crossEntropy损失求导公式推导

©️2020 CSDN 皮肤主题: Age of Ai 设计师: meimeiellie

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、C币套餐、付费专栏及课程。