矩阵求导之四：定义篇（下）

6 矩阵对标量求导

6.1 定义

对于$m\times n$的矩阵
$${\mathbf{F}}_{m\times n}\left(x\right)={\left[\begin{array}{cccc}{f}_{11}\left(x\right)& f_{12}\left(x\right)& \cdots & f_{1n}\left(x\right)\\ & & & \\ f_{21}\left(x\right)& f_{22}\left(x\right)& \cdots & f_{2n}\left(x\right)\\ & & & \\ ⋮& ⋮& \ddots & ⋮\\ & & & \\ f_{m1}\left(x\right)& f_{m2}\left(x\right)& \cdots & f_{mn}\left(x\right)\end{array}\right]}_{m\times n}$$
逐元素求导，得：
$$\frac{\partial \mathbf{F}}{\partial x}=\left[\begin{array}{cccc}\frac{\partial f_{11}}{\partial x}& \frac{\partial f_{12}}{\partial x}& \cdots & \frac{\partial f_{1n}}{\partial x}\\ & & & \\ \frac{\partial f_{21}}{\partial x}& \frac{\partial f_{22}}{\partial x}& \cdots & \frac{\partial f_{2n}}{\partial x}\\ & & & \\ ⋮& ⋮& \ddots & ⋮\\ & & & \\ \frac{\partial f_{m1}}{\partial x}& \frac{\partial f_{m2}}{\partial x}& \cdots & \frac{\partial f_{mn}}{\partial x}\end{array}\right]$$

6.2 运算法则

$$\frac{\partial \left[\mathbf{A}\left(t\right)\pm \mathbf{B}\left(t\right)\right]}{\partial t}=\frac{\partial \mathbf{A}\left(t\right)}{\partial t}\pm \frac{\partial \mathbf{B}\left(t\right)}{\partial t}$$
$$\frac{\partial \left[\lambda \left(t\right)\cdot \mathbf{A}\left(t\right)\right]}{\partial t}=\frac{\partial \lambda \left(t\right)}{\partial t}\cdot \mathbf{A}\left(t\right)+\lambda \left(t\right)\cdot \frac{\partial \mathbf{A}\left(t\right)}{\partial t}$$

其中$\lambda \left(t\right)$为变量$t$的数量函数。
$$\frac{\partial \left[\mathbf{A}\left(t\right)\cdot \mathbf{B}\left(t\right)\right]}{\partial t}=\frac{\partial \mathbf{A}\left(t\right)}{\partial t}\cdot \mathbf{B}\left(t\right)+\mathbf{A}\left(t\right)\cdot \frac{\partial \mathbf{B}\left(t\right)}{\partial t}$$

6.3 示例

【例6.1】求${\mathbf{x}}^T\mathbf{Ax}$对$t$的导数，其中
$$\begin{gathered} \mathbf{x}={\left[\begin{array}{cccc}{x}_1\left(t\right)& x_2\left(t\right)& \cdots & x_n\left(t\right)\end{array}\right]}^T \\ \mathbf{A}=\left[\begin{array}{cccc}{a}_{11}& a_{12}& \cdots & a_{1n}\\ a_{12}& a_{11}& \cdots & a_{2n}\\ ⋮& ⋮& \ddots & ⋮\\ a_{1n}& a_{2n}& \cdots & a_{nn}\end{array}\right] \end{gathered}$$
$\mathbf{A}$为对称阵。
【解】
$$\begin{array}{rl}\frac{\partial \left({\mathbf{x}}^T\mathbf{Ax}\right)}{\partial t}& =\frac{\partial {\mathbf{x}}^T}{\partial t}\cdot \mathbf{Ax}+{\mathbf{x}}^T\cdot \frac{\partial \left(\mathbf{Ax}\right)}{\partial t}\\ & \\ & =\frac{\partial {\mathbf{x}}^T}{\partial t}\cdot \mathbf{Ax}+{\mathbf{x}}^T\cdot \left(\frac{\partial \mathbf{A}}{\partial t}\cdot \mathbf{x}+\mathbf{A}\frac{\partial \mathbf{x}}{\partial t}\right)\\ & \\ & ={\hat{\mathbf{x}}}^T\mathbf{Ax}+{\mathbf{x}}^T\mathbf{A}\hat{\mathbf{x}}={\left({\hat{\mathbf{x}}}^T\mathbf{Ax}\right)}^T+{\mathbf{x}}^T\mathbf{A}\hat{\mathbf{x}}\\ & \\ & ={\mathbf{x}}^T\mathbf{A}\hat{\mathbf{x}}+{\mathbf{x}}^T\mathbf{A}\hat{\mathbf{x}}=2{\mathbf{x}}^T\mathbf{A}\hat{\mathbf{x}}\end{array}$$
【注】${\mathbf{x}}^T\mathbf{A}\hat{\mathbf{x}}$与${\hat{\mathbf{x}}}^T\mathbf{Ax}$都是数量函数，且$\mathbf{A}$为对称阵，他们等于自己的转置。
【例6.2】证明：
$$\frac{\partial \left[\mathbf{A}\left(t\right)\cdot \mathbf{B}\left(t\right)\right]}{\partial t}=\frac{\partial \mathbf{A}\left(t\right)}{\partial t}\cdot \mathbf{B}\left(t\right)+\mathbf{A}\left(t\right)\cdot \frac{\partial \mathbf{B}\left(t\right)}{\partial t}$$
【证】
设$\mathbf{A}\left(t\right)$和$\mathbf{B}\left(t\right)$分别为$n\times m$和$m\times l$矩阵：
$$\mathbf{A}\left(t\right)=\left[\begin{array}{cccc}{a}_{11}\left(t\right)& a_{12}\left(t\right)& \cdots & a_{1m}\left(t\right)\\ & & & \\ a_{21}\left(t\right)& a_{22}\left(t\right)& \cdots & a_{2m}\left(t\right)\\ & & & \\ ⋮& ⋮& \ddots & ⋮\\ & & & \\ a_{n1}\left(t\right)& a_{n2}\left(t\right)& \cdots & a_{nm}\left(t\right)\end{array}\right]=\left[\begin{array}{c}{\mathbf{\alpha }}^T\left(t\right)\\ \\ {\mathbf{\alpha }}^T\left(t\right)\\ \\ ⋮\\ \\ {\mathbf{\alpha }}^T\left(t\right)\end{array}\right]$$
$$\mathbf{B}\left(t\right)=\left[\begin{array}{cccc}{b}_{11}\left(t\right)& b_{12}\left(t\right)& \cdots & b_{1l}\left(t\right)\\ & & & \\ b_{21}\left(t\right)& b_{22}\left(t\right)& \cdots & b_{2l}\left(t\right)\\ & & & \\ ⋮& ⋮& \ddots & ⋮\\ & & & \\ b_{m1}\left(t\right)& b_{m2}\left(t\right)& \cdots & b_{ml}\left(t\right)\end{array}\right]=\left[\begin{array}{cccc}{\mathbf{\beta }}_1\left(t\right)& {\mathbf{\beta }}_2\left(t\right)& \cdots & {\mathbf{\beta }}_l\left(t\right)\end{array}\right]$$
$$\begin{array}{rl}\mathbf{A}\left(t\right)\cdot \mathbf{B}\left(t\right)& =\left[\begin{array}{cccc}{{\mathbf{\alpha }}_1}^T\left(t\right){\mathbf{\beta }}_1\left(t\right)& {{\mathbf{\alpha }}_1}^T\left(t\right){\mathbf{\beta }}_2\left(t\right)& \cdots & {{\mathbf{\alpha }}_1}^T\left(t\right){\mathbf{\beta }}_l\left(t\right)\\ & & & \\ {{\mathbf{\alpha }}_2}^T\left(t\right){\mathbf{\beta }}_1\left(t\right)& {{\mathbf{\alpha }}_2}^T\left(t\right){\mathbf{\beta }}_2\left(t\right)& \cdots & {{\mathbf{\alpha }}_2}^T\left(t\right){\mathbf{\beta }}_l\left(t\right)\\ & & & \\ ⋮& ⋮& \ddots & ⋮\\ & & & \\ {{\mathbf{\alpha }}_n}^T\left(t\right){\mathbf{\beta }}_1\left(t\right)& {{\mathbf{\alpha }}_n}^T\left(t\right){\mathbf{\beta }}_2\left(t\right)& \cdots & {{\mathbf{\alpha }}_n}^T\left(t\right){\mathbf{\beta }}_l\left(t\right)\end{array}\right]\\ & \\ & ={\left[{{\mathbf{\alpha }}_i}^T\left(t\right){\mathbf{\beta }}_j\left(t\right)\right]}_{n\times l}\end{array}$$
因此：
$$\begin{array}{rl}\frac{\partial \left[\mathbf{A}\left(t\right)\cdot \mathbf{B}\left(t\right)\right]}{\partial t}& =\frac{\partial {\left[{{\mathbf{\alpha }}_i}^T\left(t\right){\mathbf{\beta }}_j\left(t\right)\right]}_{n\times l}}{\partial t}\\ & \\ & ={\left[\frac{\partial {{\mathbf{\alpha }}_i}^T\left(t\right)}{\partial t}\cdot {\mathbf{\beta }}_j\left(t\right)+{{\mathbf{\alpha }}_i}^T\left(t\right)\cdot \frac{\partial {\mathbf{\beta }}_j\left(t\right)}{\partial t}\right]}_{n\times l}\\ & \\ & =\frac{\partial \mathbf{A}\left(t\right)}{\partial t}\cdot \mathbf{B}\left(t\right)+\mathbf{A}\left(t\right)\cdot \frac{\partial \mathbf{B}\left(t\right)}{\partial t}\end{array}$$

7 矩阵对向量求导

7.1 定义

对于$m\times l$的数量函数矩阵
$${\mathbf{F}}_{m\times n}\left(\mathbf{x}\right)={\left[\begin{array}{cccc}{f}_{11}\left(\mathbf{x}\right)& f_{12}\left(\mathbf{x}\right)& \cdots & f_{1l}\left(\mathbf{x}\right)\\ & & & \\ f_{21}\left(\mathbf{x}\right)& f_{22}\left(\mathbf{x}\right)& \cdots & f_{2l}\left(\mathbf{x}\right)\\ & & & \\ ⋮& ⋮& \ddots & ⋮\\ & & & \\ f_{m1}\left(\mathbf{x}\right)& f_{m2}\left(\mathbf{x}\right)& \cdots & f_{ml}\left(\mathbf{x}\right)\end{array}\right]}_{m\times l}$$
$n$维列向量$\mathbf{x}={\left[x_1,x_2,\cdots ,x_n\right]}^T$，有：
$$\frac{\partial \mathbf{F}\left(\mathbf{x}\right)}{\partial \mathbf{x}}={\left[\begin{array}{c}\frac{\partial \mathbf{F}\left(\mathbf{x}\right)}{\partial x_1}\\ \\ \frac{\partial \mathbf{F}\left(\mathbf{x}\right)}{\partial x_2}\\ \\ ⋮\\ \\ \frac{\partial \mathbf{F}\left(\mathbf{x}\right)}{\partial x_n}\end{array}\right]}_{nm\times l}$$
$$\frac{\partial \mathbf{F}\left(\mathbf{x}\right)}{\partial {\mathbf{x}}^T}={\left[\begin{array}{cccc}\frac{\partial \mathbf{F}\left(\mathbf{x}\right)}{\partial x_1}& \frac{\partial \mathbf{F}\left(\mathbf{x}\right)}{\partial x_2}& \cdots & \frac{\partial \mathbf{F}\left(\mathbf{x}\right)}{\partial x_n}\end{array}\right]}_{m\times ln}$$
其中：
$$\frac{\partial \mathbf{F}\left(\mathbf{x}\right)}{\partial x_i}={\left[\begin{array}{cccc}\frac{\partial f_{11}}{\partial x_i}& \frac{\partial f_{12}}{\partial x_i}& \cdots & \frac{\partial f_{1l}}{\partial x_i}\\ & & & \\ \frac{\partial f_{21}}{\partial x_i}& \frac{\partial f_{22}}{\partial x_i}& \cdots & \frac{\partial f_{2l}}{\partial x_i}\\ & & & \\ ⋮& ⋮& \ddots & ⋮\\ & & & \\ \frac{\partial f_{m1}}{\partial x_i}& \frac{\partial f_{m2}}{\partial x_i}& \cdots & \frac{\partial f_{ml}}{\partial x_i}\end{array}\right]}_{m\times l}$$

7.2 运算法则

7.2.1 加法运算

$$\frac{\partial \left[\mathbf{A}\left(\mathbf{x}\right)\pm \mathbf{B}\left(\mathbf{x}\right)\right]}{\partial \mathbf{x}}=\frac{\partial \mathbf{A}\left(\mathbf{x}\right)}{\partial \mathbf{x}}\pm \frac{\partial \mathbf{B}\left(\mathbf{x}\right)}{\partial \mathbf{x}}$$

7.2.2 数乘运算

$$\frac{\partial \left(\mathbf{\lambda A}\right)}{\partial \mathbf{x}}=\left[\frac{\partial \mathbf{\lambda }}{\partial \mathbf{x}}\right]\cdot \mathbf{A}+\mathbf{\lambda }\cdot \frac{\partial \mathbf{A}}{\partial \mathbf{x}}$$
其中：
$$\left[\frac{\partial \mathbf{\lambda }}{\partial \mathbf{x}}\right]\cdot \mathbf{A}={\left[\begin{array}{cccc}\frac{\partial \mathbf{\lambda }}{\partial x_1}\cdot \mathbf{A}& \frac{\partial \mathbf{\lambda }}{\partial x_2}\cdot \mathbf{A}& \cdots & \frac{\partial \mathbf{\lambda }}{\partial x_n}\end{array}\cdot \mathbf{A}\right]}^T$$

7.2.3 乘法运算

$$\frac{\partial \left(\mathbf{AB}\right)}{\partial \mathbf{x}}=\frac{\partial \mathbf{A}}{\partial \mathbf{x}}\cdot \mathbf{B}+\mathbf{A}\cdot \frac{\partial \mathbf{B}}{\partial \mathbf{x}}$$
其中：
$$\mathbf{A}\cdot \frac{\partial \mathbf{B}}{\partial \mathbf{x}}={\left[\begin{array}{cccc}\mathbf{A}\cdot \frac{\partial \mathbf{B}}{\partial x_1}& \mathbf{A}\cdot \frac{\partial \mathbf{B}}{\partial x_2}& \cdots & \mathbf{A}\cdot \frac{\partial \mathbf{B}}{\partial x_n}\end{array}\right]}^T$$

8 矩阵对矩阵求导

8.1 定义

对于$n\times l$的数量函数矩阵
$${\mathbf{F}}_{n\times l}\left(\mathbf{X}\right)={\left[\begin{array}{cccc}{f}_{11}\left(\mathbf{X}\right)& f_{12}\left(\mathbf{X}\right)& \cdots & f_{1l}\left(\mathbf{X}\right)\\ & & & \\ f_{21}\left(\mathbf{X}\right)& f_{22}\left(\mathbf{X}\right)& \cdots & f_{2l}\left(\mathbf{X}\right)\\ & & & \\ ⋮& ⋮& \ddots & ⋮\\ & & & \\ f_{n1}\left(\mathbf{X}\right)& f_{n2}\left(\mathbf{X}\right)& \cdots & f_{nl}\left(\mathbf{X}\right)\end{array}\right]}_{n\times l}$$
对于$p\times m$的数量矩阵
$${\mathbf{X}}_{p\times m}=\left[\begin{array}{cccc}{x}_{11}& x_{12}& \cdots & x_{1m}\\ x_{21}& x_{22}& \cdots & x_{2m}\\ ⋮& ⋮& \ddots & ⋮\\ x_{p1}& x_{p2}& \cdots & x_{pm}\end{array}\right]$$
有：
$$\frac{\partial \mathbf{F}\left(\mathbf{X}\right)}{\partial \mathbf{X}}=\left[\begin{array}{cccc}\frac{\partial \mathbf{F}\left(\mathbf{X}\right)}{\partial x_{11}}& \frac{\partial \mathbf{F}\left(\mathbf{X}\right)}{\partial x_{12}}& \cdots & \frac{\partial \mathbf{F}\left(\mathbf{X}\right)}{\partial x_{1m}}\\ & & & \\ \frac{\partial \mathbf{F}\left(\mathbf{X}\right)}{\partial x_{21}}& \frac{\partial \mathbf{F}\left(\mathbf{X}\right)}{\partial x_{22}}& \cdots & \frac{\partial \mathbf{F}\left(\mathbf{X}\right)}{\partial x_{2m}}\\ & & & \\ ⋮& ⋮& \ddots & ⋮\\ & & & \\ \frac{\partial \mathbf{F}\left(\mathbf{X}\right)}{\partial x_{p1}}& \frac{\partial \mathbf{F}\left(\mathbf{X}\right)}{\partial x_{p2}}& \cdots & \frac{\partial \mathbf{F}\left(\mathbf{X}\right)}{\partial x_{pm}}\end{array}\right]$$
对于每个分块矩阵，有：
$$\left[\frac{\partial \mathbf{F}\left(\mathbf{x}\right)}{\partial x_{ij}}\right]=\left[\begin{array}{cccc}\frac{\partial f_{11}\left(\mathbf{X}\right)}{\partial x_{ij}}& \frac{\partial f_{12}\left(\mathbf{X}\right)}{\partial x_{ij}}& \cdots & \frac{\partial f_{1l}\left(\mathbf{X}\right)}{\partial x_{ij}}\\ & & & \\ \frac{\partial f_{21}\left(\mathbf{X}\right)}{\partial x_{ij}}& \frac{\partial f_{22}\left(\mathbf{X}\right)}{\partial x_{ij}}& \cdots & \frac{\partial f_{2l}\left(\mathbf{X}\right)}{\partial x_{ij}}\\ & & & \\ ⋮& ⋮& \ddots & ⋮\\ & & & \\ \frac{\partial f_{n1}\left(\mathbf{X}\right)}{\partial x_{ij}}& \frac{\partial f_{n2}\left(\mathbf{X}\right)}{\partial x_{ij}}& \cdots & \frac{\partial f_{nl}\left(\mathbf{X}\right)}{\partial x_{ij}}\end{array}\right]$$
易知$\frac{\partial \mathbf{F}\left(\mathbf{X}\right)}{\partial \mathbf{X}}$的结果为$pn\times ml$维矩阵。

参考文献

[1] 向量对矩阵求导
[2] 向量，标量对向量求导数