深度学习笔记之循环神经网络(四)铺垫：Softmax函数的反向传播过程

引言

上一节介绍了循环神经网络前馈计算过程的基本逻辑，以及作为语言模型时，衡量一个语言模型的优劣性指标——困惑度。本节介绍$\text{Softmax}$函数的反向传播$(\text{Backward Propagation,BP})$过程。

铺垫：$\text{Softmax}$的反向传播过程

场景构建

假设一个$\mathcal{L}$层全连接神经网络用作$\mathcal{C}$分类的分类任务，并且已知由$m$个训练样本构成的训练集$\mathcal{D}$：
$$\mathcal{D}=\{(x^{(i)},y^{(i)}){\}}_{i=1}^m$$
中间的计算过程忽略。仅观察输出结果。设每一个$x^{(i)}(i=1,2,\cdots ,m)$的对应预测结果为${\hat{y}}^{(i)}$，使用交叉熵$(\text{CrossEntropy})$对其计算损失：
$$\mathcal{L}\left[y^{(i)},{\hat{y}}^{(i)}\right]=-\sum _{j=1}^{\mathcal{C}}{y}_j^{(i)}\log {\hat{y}}_j^{(i)}$$
相应地，对训练集$\mathcal{D}$的损失函数$\mathcal{J}(\mathcal{W})$表示为：
这里将偏置项$b$忽略掉了。
$$\mathcal{J}(\mathcal{W})=\frac{1}{m}\sum _{i=1}^m\mathcal{L}\left[y^{(i)},{\hat{y}}^{(i)}\right]$$

关于最后一层神经网络输出${\mathcal{Z}}^{(\mathcal{L})}$与$\text{Softmax}$激活函数的前馈计算过程表示如下：
$$\hat{y}=a^{(\mathcal{L})}=\text{Softmax}({\mathcal{Z}}^{(\mathcal{L})})$$

$\text{Softmax}$反向传播过程

以单个样本$(x,y)\in \mathcal{D}$为例。首先计算该样本的损失函数结果$\mathcal{L}(y,\hat{y})$关于预测输出$\hat{y}=a^{(\mathcal{L})}$ 的导数结果：
$$\begin{array}{rl}\frac{\partial \mathcal{L}}{\partial a^{(\mathcal{L})}}& =\frac{\partial }{\partial a^{(\mathcal{L})}}\left[-\sum _{j=1}^{\mathcal{C}}{y}_j\log {\hat{y}}_j\right]\\ & =\frac{\partial }{\partial a^{(\mathcal{L})}}\left[-(y_1\log {\hat{y}}_1+y_2\log {\hat{y}}_2+\cdots +y_{\mathcal{C}}\log {\hat{y}}_{\mathcal{C}})\right]\\ & =\frac{\partial }{\partial a^{(\mathcal{L})}}\left[-(y_1\log a_1^{(\mathcal{L})}+y_2\log a_2^{(\mathcal{L})}+\cdots +y_{\mathcal{C}}\log a_{\mathcal{C}}^{(\mathcal{L})})\right]\end{array}$$
很明显，$\mathcal{L}$表示各维度的连加和，是一个标量；而此时的$a^{(\mathcal{L})}$是一个$1\times \mathcal{C}$的向量。其求导结果表示如下：
标量对向量求导见文章末尾链接，侵删。
$$\begin{array}{rl}\frac{\partial \mathcal{L}}{\partial a^{(\mathcal{L})}}& =\left[\frac{\partial \mathcal{L}}{\partial a_1^{(\mathcal{L})}},\cdots ,\frac{\partial \mathcal{L}}{\partial a_{\mathcal{C}}^{(\mathcal{L})}}\right]\\ & =\left\{\frac{\partial }{\partial a_1^{(\mathcal{L})}}\left[-(\underset{a_1^{(\mathcal{L})相关}}{\underbrace{y_1\log a_1^{(\mathcal{L})}}}+\underset{a_1^{(\mathcal{L})无关}}{\underbrace{\cdots +y_{\mathcal{C}}\log a_{\mathcal{C}}^{(\mathcal{L})}}})\right],\cdots ,\frac{\partial }{\partial a_{\mathcal{C}}^{(\mathcal{L})}}\left[-(\underset{a_{\mathcal{C}}^{(\mathcal{L})无关}}{\underbrace{y_1\log a_1^{(\mathcal{L})}+\cdots }}+\underset{a_{\mathcal{C}}^{(\mathcal{L})相关}}{\underbrace{y_{\mathcal{C}}\log a_{\mathcal{C}}^{(\mathcal{L})}}})\right]\right\}\\ & =\left[-\frac{y_1}{a_1^{(\mathcal{L})}},\cdots ,-\frac{y_{\mathcal{C}}}{a_{\mathcal{C}}^{(\mathcal{L})}}\right]\\ & =-\frac{(y_1,\cdots ,y_{\mathcal{C}})}{\left(a_1^{(\mathcal{L})},\cdots ,a_{\mathcal{C}}^{(\mathcal{L})}\right)}\\ & =-\frac{y}{\hat{y}}\end{array}$$
继续向前传播，计算$\begin{array}{r}\frac{\partial \mathcal{L}}{\partial {\mathcal{Z}}^{(\mathcal{L})}}\end{array}$：
$$\frac{\partial \mathcal{L}}{\partial {\mathcal{Z}}^{(\mathcal{L})}}=\frac{\partial \mathcal{L}}{\partial a^{(\mathcal{L})}}\cdot \frac{\partial a^{(\mathcal{L})}}{\partial {\mathcal{Z}}^{(\mathcal{L})}}$$
关于$\begin{array}{r}\frac{\partial a^{(\mathcal{L})}}{\partial {\mathcal{Z}}^{(\mathcal{L})}}\end{array}$，由于$a^{(\mathcal{L})},{\mathcal{Z}}^{(\mathcal{L})}$均是$1\times \mathcal{C}$的向量。其导数结果表示如下：
这是一个$\mathcal{C}\times \mathcal{C}\times 1$的三维张量。
$$\begin{array}{rl}\frac{\partial a^{(\mathcal{L})}}{\partial {\mathcal{Z}}^{(\mathcal{L})}}& ={\left[\frac{\partial a^{(\mathcal{L})}}{\partial z_1^{(\mathcal{L})}},\cdots ,\frac{\partial a^{(\mathcal{L})}}{\partial z_{\mathcal{C}}^{(\mathcal{L})}}\right]}_{\mathcal{C}\times \mathcal{C}\times 1}^T\\ & ={\left\{\frac{\partial }{\partial z_1^{(\mathcal{L})}}\left[\frac{\exp ({\mathcal{Z}}^{(\mathcal{L})})}{{\sum }_{i=1}^{\mathcal{C}}\exp (z_i^{(\mathcal{L})})}\right],\cdots ,\frac{\partial }{\partial z_{\mathcal{C}}^{(\mathcal{L})}}\left[\frac{\exp ({\mathcal{Z}}^{(\mathcal{L})})}{{\sum }_{i=1}^{\mathcal{C}}\exp (z_{\mathcal{C}}^{(\mathcal{L})})}\right]\right\}}_{\mathcal{C}\times \mathcal{C}\times 1}^T\end{array}$$
这里以第一项为例，不可否认的是，它是一个$1\times \mathcal{C}$的向量结果。并且$z_1^{(\mathcal{L})}$是一个标量，它的导数结果表示如下：
其中$\begin{array}{r}\frac{\exp ({\mathcal{Z}}^{(\mathcal{L})})}{{\sum }_{i=1}^{\mathcal{C}}\exp (z_i^{(\mathcal{L})})}\end{array}$是输出结果$a^{(\mathcal{L})}$的第一个分量。记作$a_1^{(\mathcal{L})}$.
$$\frac{\partial }{\partial z_1^{(\mathcal{L})}}\left[\frac{\exp ({\mathcal{Z}}^{(\mathcal{L})})}{{\sum }_{i=1}^{\mathcal{C}}\exp (z_i^{(\mathcal{L})})}\right]={\left\{\frac{\partial }{\partial z_1^{(\mathcal{L})}}\underset{a_1^{(\mathcal{L})}}{\underbrace{\left[\frac{\exp (z_1^{(\mathcal{L})})}{{\sum }_{i=1}^{\mathcal{C}}\exp (z_i^{(\mathcal{L})})}\right]}},\cdots ,\frac{\partial }{\partial z_1^{(\mathcal{L})}}\underset{a_{\mathcal{C}}^{(\mathcal{L})}}{\underbrace{\left[\frac{\exp (z_{\mathcal{C}}^{(\mathcal{L})})}{{\sum }_{i=1}^{\mathcal{C}}\exp (z_i^{(\mathcal{L})})}\right]}}\right\}}_{1\times \mathcal{C}}$$
继续以第一项为例，关于$\begin{array}{r}\frac{\partial a_1^{(\mathcal{L})}}{\partial z_1^{(\mathcal{L})}}\end{array}$结果表示如下：
除法求导~
其中${\left[{\sum }_{i=1}^{\mathcal{C}}\exp (z_i^{(\mathcal{L})})\right]}^{\prime }$中与$z_1^{(\mathcal{L})}$相关的只有第一项。因此该项结果为:$\exp (z_i^{(L)})$.
$$\begin{array}{rl}\frac{\partial a_1^{(\mathcal{L})}}{\partial z_1^{(\mathcal{L})}}& =\frac{\partial }{\partial z_1^{(\mathcal{L})}}\left[\frac{\exp ({\mathcal{Z}}^{(\mathcal{L})})}{{\sum }_{i=1}^{\mathcal{C}}\exp (z_i^{(\mathcal{L})})}\right]\\ & =\frac{{\left[\exp (z_1^{(\mathcal{L})})\right]}^{\prime }\cdot {\sum }_{i=1}^{\mathcal{C}}\exp (z_i^{(\mathcal{L})})-\exp (z_1^{(\mathcal{L})})\cdot {\left[{\sum }_{i=1}^{\mathcal{C}}\exp (z_i^{(\mathcal{L})})\right]}^{\prime }}{{\left[{\sum }_{i=1}^{\mathcal{C}}\exp (z_i^{(\mathcal{L})})\right]}^2}\\ & =\frac{\exp (z_1^{(\mathcal{L})})\cdot {\sum }_{i=1}^{\mathcal{C}}\exp (z_i^{(\mathcal{L})})-{\left[\exp (z_1^{(L)})\right]}^2}{{\left[{\sum }_{i=1}^{\mathcal{C}}\exp (z_i^{(\mathcal{L})})\right]}^2}\end{array}$$
分子提出$\exp (z_1^{(\mathcal{L})})$，分母平方项展开：
$$\begin{array}{rl}\frac{\partial a_1^{(\mathcal{L})}}{\partial z_1^{(\mathcal{L})}}& =\frac{\exp (z_1^{(\mathcal{L})})\cdot \left[{\sum }_{i=1}^{\mathcal{C}}\exp (z_i^{(\mathcal{L})})-\exp (z_1^{(\mathcal{L})})\right]}{{\left[{\sum }_{i=1}^{\mathcal{C}}\exp (z_i^{(\mathcal{L})})\right]}^2}\\ & =\frac{\exp (z_1^{(\mathcal{L})})}{{\sum }_{i=1}^{\mathcal{C}}\exp (z_i^{(\mathcal{L})})}\cdot \frac{{\sum }_{i=1}^{\mathcal{C}}\exp (z_i^{(\mathcal{L})})-\exp (z_1^{(\mathcal{L})})}{{\sum }_{i=1}^{\mathcal{C}}\exp (z_i^{(\mathcal{L})})}\\ & =\frac{\exp (z_1^{(\mathcal{L})})}{{\sum }_{i=1}^{\mathcal{C}}\exp (z_i^{(\mathcal{L})})}\cdot \left[1-\frac{\exp (z_1^{(\mathcal{L})})}{{\sum }_{i=1}^{\mathcal{C}}\exp (z_i^{(\mathcal{L})})}\right]\\ & =a_1^{(\mathcal{L})}\cdot (1-a_1^{(\mathcal{L})})\end{array}$$

同理，关于两个下标参数$p,q$；当$p=q$时，有：
∂ a q ( L ) ∂ z p ( L ) = a p ( L ) ⋅ ( 1 − a p ( L ) ) p , q ∈ { 1 , 2 , ⋯   , C } ; p = q \frac{\partial a_q^{(\mathcal L)}}{\partial z_p^{(\mathcal L)}} = a_p^{(\mathcal L)} \cdot (1 - a_p^{(\mathcal L)}) \quad p,q \in \{1,2,\cdots,\mathcal C\};p = q ∂zp(L)​∂aq(L)​​=ap(L)​⋅(1−ap(L)​)p,q∈{1,2,⋯,C};p=q
当$p\ne q$时，对应结果表示为：
其中$\begin{array}{r}{\left[\frac{\partial \exp (z_q^{(\mathcal{L})})}{\partial z_p^{(\mathcal{L})}}\right]}_{p\ne q}=0\end{array}$恒成立。
$$\begin{array}{rl}\frac{\partial a_q^{(\mathcal{L})}}{\partial z_p^{(\mathcal{L})}}& =\frac{0\cdot {\sum }_{i=1}^{\mathcal{C}}\exp (z_i^{(\mathcal{L})})-\exp (z_q^{(\mathcal{L})})\cdot \exp (z_p^{(\mathcal{L})})}{{\left[{\sum }_{i=1}^{\mathcal{C}}\exp (z_i^{(\mathcal{L})})\right]}^2}\\ & =-\frac{exp(z_q^{(\mathcal{L})})}{{\sum }_{i=1}^{\mathcal{C}}\exp (z_i^{(\mathcal{L})})}\cdot \frac{\exp (z_p^{(\mathcal{L})})}{{\sum }_{i=1}^{\mathcal{C}}\exp (z_i^{(\mathcal{L})})}\\ & =-a_p^{(\mathcal{L})}\cdot a_q^{(\mathcal{L})}\end{array}$$
至此，$\begin{array}{r}{\left[\frac{\partial a^{(\mathcal{L})}}{\partial {\mathcal{Z}}^{(\mathcal{L})}}\right]}_{\mathcal{C}\times \mathcal{C}\times 1}\end{array}$中的所有项均可进行表示。将该三维张量进行压缩(删除最后一个维度)，可以得到一个雅可比矩阵$(\text{Jacobian Matrix})$：
矩阵中的每一个元素均可使用上述两种方式进行表达。
$$\frac{\partial a^{(\mathcal{L})}}{\partial {\mathcal{Z}}^{(\mathcal{L})}}={\left[\begin{array}{cccc}\begin{array}{r}\frac{\partial a_1^{(\mathcal{L})}}{\partial z_1^{(\mathcal{L})}}\end{array}& \begin{array}{r}\frac{\partial a_1^{(\mathcal{L})}}{\partial z_2^{(\mathcal{L})}}\end{array}& \cdots & \begin{array}{r}\frac{\partial a_1^{(\mathcal{L})}}{\partial z_{\mathcal{C}}^{(\mathcal{L})}}\end{array}\\ \begin{array}{r}\frac{\partial a_2^{(\mathcal{L})}}{\partial z_1^{(\mathcal{L})}}\end{array}& \begin{array}{r}\frac{\partial a_2^{(\mathcal{L})}}{\partial z_2^{(\mathcal{L})}}\end{array}& \cdots & \begin{array}{r}\frac{\partial a_2^{(\mathcal{L})}}{\partial z_{\mathcal{C}}^{(\mathcal{L})}}\end{array}\\ ⋮& ⋮& \ddots & ⋮\\ \begin{array}{r}\frac{\partial a_{\mathcal{C}}^{(\mathcal{L})}}{\partial z_1^{(\mathcal{L})}}\end{array}& \begin{array}{r}\frac{\partial a_{\mathcal{C}}^{(\mathcal{L})}}{\partial z_2^{(\mathcal{L})}}\end{array}& \cdots & \begin{array}{r}\frac{\partial a_{\mathcal{C}}^{(\mathcal{L})}}{\partial z_{\mathcal{C}}^{(\mathcal{L})}}\end{array}\end{array}\right]}_{\mathcal{C}\times \mathcal{C}}$$
此时，对$\begin{array}{r}\frac{\partial \mathcal{L}}{\partial {\mathcal{Z}}^{(\mathcal{L})}}\end{array}$进行表达：
其结果是一个$1\times \mathcal{C}$的向量格式。
$$\begin{array}{rl}\frac{\partial \mathcal{L}}{\partial a^{(\mathcal{L})}}\cdot \frac{\partial a^{(\mathcal{L})}}{\partial {\mathcal{Z}}^{(\mathcal{L})}}& =\left[-\frac{y_1}{a_1^{(\mathcal{L})}},\cdots ,-\frac{y_{\mathcal{C}}}{a_{\mathcal{C}}^{(\mathcal{L})}}\right]\cdot {\left[\begin{array}{cccc}\begin{array}{r}\frac{\partial a_1^{(\mathcal{L})}}{\partial z_1^{(\mathcal{L})}}\end{array}& \begin{array}{r}\frac{\partial a_1^{(\mathcal{L})}}{\partial z_2^{(\mathcal{L})}}\end{array}& \cdots & \begin{array}{r}\frac{\partial a_1^{(\mathcal{L})}}{\partial z_{\mathcal{C}}^{(\mathcal{L})}}\end{array}\\ \begin{array}{r}\frac{\partial a_2^{(\mathcal{L})}}{\partial z_1^{(\mathcal{L})}}\end{array}& \begin{array}{r}\frac{\partial a_2^{(\mathcal{L})}}{\partial z_2^{(\mathcal{L})}}\end{array}& \cdots & \begin{array}{r}\frac{\partial a_2^{(\mathcal{L})}}{\partial z_{\mathcal{C}}^{(\mathcal{L})}}\end{array}\\ ⋮& ⋮& \ddots & ⋮\\ \begin{array}{r}\frac{\partial a_{\mathcal{C}}^{(\mathcal{L})}}{\partial z_1^{(\mathcal{L})}}\end{array}& \begin{array}{r}\frac{\partial a_{\mathcal{C}}^{(\mathcal{L})}}{\partial z_2^{(\mathcal{L})}}\end{array}& \cdots & \begin{array}{r}\frac{\partial a_{\mathcal{C}}^{(\mathcal{L})}}{\partial z_{\mathcal{C}}^{(\mathcal{L})}}\end{array}\end{array}\right]}_{\mathcal{C}\times \mathcal{C}}\\ & ={\left[-\sum _{i=1}^{\mathcal{C}}\frac{y_i}{a_i^{(\mathcal{L})}}\cdot \frac{\partial a_i^{(\mathcal{L})}}{\partial z_1^{(\mathcal{L})}},\cdots ,-\sum _{i=1}^{\mathcal{C}}\frac{y_i}{a_i^{(\mathcal{L})}}\cdot \frac{\partial a_i^{(\mathcal{L})}}{\partial z_{\mathcal{C}}^{(\mathcal{L})}}\right]}_{1\times \mathcal{C}}\\ & ={\left[-\sum _{i=1}^{\mathcal{C}}\frac{y_i}{a_i^{(\mathcal{L})}}\cdot \frac{\partial a_i^{(\mathcal{L})}}{\partial z_j^{(\mathcal{L})}}\right]}_{1\times \mathcal{C}}j=1,2,\cdots ,\mathcal{C}\end{array}$$
将 ∂ a i ( L ) ∂ z j ( L ) ( i , j ∈ { 1 , 2 , ⋯   , C } ) = { a i ( L ) ( 1 − a j ( L ) ) i = j − a i ( L ) ⋅ a j ( L ) i ≠ j \begin{aligned}\frac{\partial a_i^{(\mathcal L)}}{\partial z_j^{(\mathcal L)}}(i,j \in \{1,2,\cdots,\mathcal C\}) = \begin{cases}a_i^{(\mathcal L)}(1 - a_j^{(\mathcal L)}) \quad i = j \\ -a_i^{(\mathcal L)} \cdot a_j^{(\mathcal L)} \quad i \neq j \end{cases}\end{aligned} ∂zj(L)​∂ai(L)​​(i,j∈{1,2,⋯,C})={ai(L)​(1−aj(L)​)i=j−ai(L)​⋅aj(L)​i=j​​两种情况代入到上式中：
可以消掉$a_i^{(\mathcal{L})}$.
需要注意的是，这里的连加号${\sum }_{i=1}^{\mathcal{C}}$是均满足条件时的累加结果。如果只有一项满足条件，那么$\mathcal{C}=1$，以此类推。
$$-\sum _{i=1}^{\mathcal{C}}\frac{y_i}{a_i^{(\mathcal{L})}}\cdot \frac{\partial a_i^{(\mathcal{L})}}{\partial z_j^{(\mathcal{L})}}=\{\begin{array}{l}\begin{array}{rl}& \sum _{i=1}^{\mathcal{C}}{y}_i\cdot a_j^{(\mathcal{L})}-y_{i}i=j\\ & \sum _{i=1}^{\mathcal{C}}{y}_i\cdot a_j^{(\mathcal{L})}i\ne j\end{array}\end{array}$$
关于$\begin{array}{r}{\left[\frac{\partial \mathcal{L}}{\partial a^{(\mathcal{L})}}\cdot \frac{\partial a^{(\mathcal{L})}}{\partial {\mathcal{Z}}^{(\mathcal{L})}}\right]}_{1\times \mathcal{C}}\end{array}$中的结果，其每一项内的连加项中，只有一项是$i=j$的情况。因而对$1\times \mathcal{C}$向量中的每一项均执行如下操作：
就是分成$i=j$的$1$项与$i\ne j$的$\mathcal{C}-1$项分别运算。
其中$\begin{array}{r}\sum _{i=1}^{\mathcal{C}}{y}_i\end{array}$是真实标签向量各分量之和。而真实标签中只有 { 0 , 1 } \{0,1\} {0,1}两种元素(是该分类的为$1$,不是该分类的为$0$)因此,$\begin{array}{r}\sum _{i=1}^{\mathcal{C}}{y}_i\end{array}$ = 1.
$$\begin{array}{rl}-\sum _{i=1}^{\mathcal{C}}\frac{y_i}{a_i^{(\mathcal{L})}}\cdot \frac{\partial a_i^{(\mathcal{L})}}{\partial z_j^{(\mathcal{L})}}& =\underset{i=j}{\underbrace{-y_j+y_j\cdot a_j^{(\mathcal{L})}}}+\underset{i\ne j}{\underbrace{\sum _{i\ne j}{y}_i\cdot a_i^{(\mathcal{L})}}}\\ & =-y_j+\left(y_j\cdot a_j^{(\mathcal{L})}+\sum _{i\ne j}{y}_i\cdot a_j^{(\mathcal{L})}\right)\\ & =-y_j+a_j^{(\mathcal{L})}\cdot \sum _{i=1}^{\mathcal{C}}{y}_i\\ & =a_j^{(\mathcal{L})}-y_j\end{array}$$
这仅仅是一个分量的结果，所有分量的结果组成一个$1\times \mathcal{C}$的向量：
$${\left[a_j^{(\mathcal{L})}-y_j\right]}_{1\times \mathcal{C}}j=1,2,\cdots ,\mathcal{C}\Rightarrow a^{(\mathcal{L})}-y$$

相关参考：
向量对向量求导
关于 Softmax 回归的反向传播求导数过程