y=Cx 求导

本人数学很菜, 连求导都要想半天… 下面是记录读这篇博客的过程中, 对卷积求导的过程
https://blog.csdn.net/xiaojiajia007/article/details/75041651

假设有$y_{4\times 1}=C_{4\times 16}{x}_{16\times 1}$，已知$\frac{\partial Loss}{\mathbf{y}}$，求$\frac{\partial Loss}{\mathbf{x}}$

$\frac{\partial Loss}{\mathbf{x}}=\left[\begin{array}{c}\frac{\partial Loss}{\partial x_1}\\ ⋮\\ \frac{\partial Loss}{\partial x_{16}}\end{array}\right]=\left[\begin{array}{c}{\sum }_{i=1}^4\frac{\partial Loss}{\partial y_i}\frac{\partial y_i}{\partial x_1}\\ ⋮\\ {\sum }_{i=1}^4\frac{\partial Loss}{\partial y_i}\frac{\partial y_i}{\partial x_{16}}\end{array}\right]$

元素相乘再求和可以写成矩阵乘法:

$=\left[\begin{array}{c}\left[\begin{array}{ccc}\frac{\partial Loss}{\partial y_1}& \dots & \frac{\partial Loss}{\partial y_4}\end{array}\right]\times \left[\begin{array}{c}\frac{\partial y_1}{\partial x_1}\\ ⋮\\ \frac{\partial y_4}{\partial x_1}\end{array}\right]\\ ⋮\\ \left[\begin{array}{ccc}\frac{\partial Loss}{y_1}& \dots & \frac{\partial Loss}{\partial y_4}\end{array}\right]\times \left[\begin{array}{c}\frac{\partial y_1}{\partial x_{16}}\\ ⋮\\ \frac{\partial y_4}{\partial x_{16}}\end{array}\right]\end{array}\right]$

$=\left[\begin{array}{c}{\frac{\partial Loss}{\partial \mathbf{y}}}^T\times \frac{\partial \mathbf{y}}{\partial x_1}\\ ⋮\\ {\frac{\partial Loss}{\partial \mathbf{y}}}^T\times \frac{\partial \mathbf{y}}{\partial x_{16}}\end{array}\right]$

这种情况不能提公因式, 因为$\frac{\partial \mathbf{y}}{\partial x}$都是列向量, 把$\frac{\partial Loss}{\partial \mathbf{y}}$提出来之后, 右边不能成为矩阵

所以转置一下. $B^{T}A=(A^{T}B{)}^T$, 由于$\frac{\partial Loss}{\partial \mathbf{y}}\times \frac{\partial \mathbf{y}}{\partial x_1}$得到的是$1\times 1$的数字, 所以$B^{T}A=(A^{T}B{)}^T=A^{T}B$

$=\left[\begin{array}{c}{\frac{\partial \mathbf{y}}{\partial x_1}}^T\times \frac{\partial Loss}{\partial \mathbf{y}}\\ ⋮\\ {\frac{\partial \mathbf{y}}{x_{16}}}^T\times \frac{\partial Loss}{\partial \mathbf{y}}\end{array}\right]$

$=\left[\begin{array}{c}{\frac{\partial \mathbf{y}}{\partial x_1}}^T\\ ⋮\\ {\frac{\partial \mathbf{y}}{x_{16}}}^T\end{array}\right]\times \frac{\partial Loss}{\partial \mathbf{y}}$

$=\left[\begin{array}{ccc}\frac{\partial y_1}{\partial x_1}& \dots & \frac{\partial y_4}{\partial x_1}\\ ⋮& & ⋮\\ \frac{\partial y_1}{\partial x_{16}}& \dots & \frac{\partial y_4}{\partial x_{16}}\end{array}\right]\times \left[\begin{array}{c}\frac{\partial Loss}{\partial y_1}\\ ⋮\\ \frac{\partial Loss}{\partial y_4}\end{array}\right]$

$=\left[\begin{array}{ccc}{c}_{1,1}& \dots & c_{4,1}\\ ⋮& & ⋮\\ c_{1,16}& \dots & c_{4,16}\end{array}\right]\times \left[\begin{array}{c}\frac{\partial Loss}{\partial y_1}\\ ⋮\\ \frac{\partial Loss}{\partial y_4}\end{array}\right]$