神经网络矩阵运算公式总结

• 1.

$$tr\left(A\cdot \left(B\odot C\right)\right)=tr\left(\left(A\odot B^T\right)\cdot C\right)$$

---

• 2.

$$tr\left(A\cdot \left(B{\ast }_{valid}C\right)\right)=tr\left(\left(A{\ast }_{full}{B}_{rot}^T\right)\cdot C\right)=tr\left(\left(A{\ast }_{valid}{C}_{rot}^T\right)\cdot B\right)$$$$其中:A\in {\mathbb{R}}^{m-n+1\times m-n+1},B\in {\mathbb{R}}^{n\times n},C\in {\mathbb{R}}^{m\times m},且m\ge n\ge 2$$

---

• 3.

$$tr\left(A\cdot \left(B{\ast }_{full}C\right)\right)=tr\left(\left(A{\ast }_{valid}{B}_{rot}^T\right)\cdot C\right)=tr\left(\left(A{\ast }_{valid}{C}_{rot}^T\right)\cdot B\right)$$$$其中:A\in {\mathbb{R}}^{m+n-1\times m+n-1},B\in {\mathbb{R}}^{n\times n},C\in {\mathbb{R}}^{m\times m},且m\ge 2,n\ge 2$$

---

• 4.

$$A_{rot}^T={\left(A_{rot}\right)}^T={\left(A^T\right)}_{rot}$$

---

• 5.

d ( A ∗ { f u l l   O R   v a l i d } B ) = d A ∗ { f u l l   O R   v a l i d } B + A ∗ { f u l l   O R   v a l i d } d B d(A*_{\{full\ OR\ valid\}}B)=dA*_{\{full\ OR\ valid\}}B+A*_{\{full\ OR\ valid\}}dB d(A∗{full OR valid}​B)=dA∗{full OR valid}​B+A∗{full OR valid}​dB

---

• 6.

$$d(A\cdot B)=dA\cdot B+A\cdot dB$$

---

• 7.

$$d(A\odot B)=dA\odot B+A\odot dB$$

---

• 8.

$$df=tr\left({\frac{\partial f}{\partial X}}^T\cdot dX\right)$$

---

• 9.

$$dtr(X)=tr(dX)$$

---

• 10.

$$设A,B,C为四维数组,$$$$A\in {\mathbb{R}}^{m\times n\times h\times h},B\in {\mathbb{R}}^{m\times p\times l\times l},C\in {\mathbb{R}}^{p\times n\times k\times k},$$$$A=\left[\begin{array}{cccc}{\alpha }_{11}& {\alpha }_{12}& \dots & {\alpha }_{1n}\\ {\alpha }_{21}& {\alpha }_{22}& \dots & {\alpha }_{2n}\\ ⋮& ⋮& \ddots & ⋮\\ {\alpha }_{m1}& {\alpha }_{m2}& \dots & {\alpha }_{mn}\end{array}\right],$$$${\alpha }_{ij}\in {\mathbb{R}}^{h\times h}\left(i=1,2,\dots ,m,j=1,2,\dots ,n\right),$$$$及:{\alpha }_{ij}=\left[\begin{array}{cccc}{a}_{11}& a_{12}& \dots & a_{1h}\\ a_{21}& a_{22}& \dots & a_{2h}\\ ⋮& ⋮& \ddots & ⋮\\ a_{h1}& a_{h2}& \dots & a_{hh}\end{array}\right],$$$$B=\left[\begin{array}{cccc}{\beta }_{11}& {\beta }_{12}& \dots & {\beta }_{1p}\\ {\beta }_{21}& {\beta }_{22}& \dots & {\beta }_{2p}\\ ⋮& ⋮& \ddots & ⋮\\ {\beta }_{m1}& {\beta }_{m2}& \dots & {\beta }_{mp}\end{array}\right],$$$${\beta }_{ij}\in {\mathbb{R}}^{l\times l}\left(i=1,2,\dots ,m,j=1,2,\dots ,p\right),$$$$C=\left[\begin{array}{cccc}{\gamma }_{11}& {\gamma }_{12}& \dots & {\gamma }_{1n}\\ {\gamma }_{21}& {\gamma }_{22}& \dots & {\gamma }_{2n}\\ ⋮& ⋮& \ddots & ⋮\\ {\gamma }_{p1}& {\gamma }_{p2}& \dots & {\gamma }_{pn}\end{array}\right],$$$${\gamma }_{ij}\in {\mathbb{R}}^{k\times k}\left(i=1,2,\dots ,p,j=1,2,\dots ,n\right).$$$$A、B、C可视为元素类型为二维数组的二维数组.$$$$定义四维数组卷积运算A=B\ast C(\ast 包括{\ast }_{valid}与{\ast }_{full})为:$$$$\left[\begin{array}{cccc}{\beta }_{11}& {\beta }_{12}& \dots & {\beta }_{1p}\\ {\beta }_{21}& {\beta }_{22}& \dots & {\beta }_{2p}\\ ⋮& ⋮& \ddots & ⋮\\ {\beta }_{m1}& {\beta }_{m2}& \dots & {\beta }_{mp}\end{array}\right]\ast \left[\begin{array}{cccc}{\gamma }_{11}& {\gamma }_{12}& \dots & {\gamma }_{pn}\\ {\gamma }_{21}& {\gamma }_{22}& \dots & {\gamma }_{pn}\\ ⋮& ⋮& \ddots & ⋮\\ {\gamma }_{p1}& {\gamma }_{p2}& \dots & {\gamma }_{pn}\end{array}\right]=$$$$\left[\begin{array}{cccc}{\sum }_{i=1}^p{\beta }_{1i}\ast {\gamma }_{i1}& {\sum }_{i=1}^p{\beta }_{1i}\ast {\gamma }_{i2}& \dots & {\sum }_{i=1}^p{\beta }_{1i}\ast {\gamma }_{in}\\ {\sum }_{i=1}^p{\beta }_{2i}\ast {\gamma }_{i1}& {\sum }_{i=1}^p{\beta }_{2i}\ast {\gamma }_{i2}& \dots & {\sum }_{i=1}^p{\beta }_{2i}\ast {\gamma }_{in}\\ ⋮& ⋮& \ddots & ⋮\\ {\sum }_{i=1}^p{\beta }_{mi}\ast {\gamma }_{i1}& {\sum }_{i=1}^p{\beta }_{mi}\ast {\gamma }_{i2}& \dots & {\sum }_{i=1}^p{\beta }_{mi}\ast {\gamma }_{in}\end{array}\right]$$$$例如:\left[\begin{array}{cc}{a}_{11}& a_{12}\\ a_{21}& a_{22}\end{array}\right]\ast \left[\begin{array}{cc}{b}_{11}& b_{12}\\ b_{21}& b_{22}\end{array}\right]=$$$$\left[\begin{array}{cc}{a}_{11}\ast b_{11}+a_{12}\ast b_{21}& a_{11}\ast b_{12}+a_{12}\ast b_{22}\\ a_{21}\ast b_{11}+a_{22}\ast b_{21}& a_{21}\ast b_{12}+a_{22}\ast b_{22}\end{array}\right]$$

---

• 11.

$$对于四维数组$$$$A=\left[\begin{array}{cccc}{\alpha }_{11}& {\alpha }_{12}& \dots & {\alpha }_{1n}\\ {\alpha }_{21}& {\alpha }_{22}& \dots & {\alpha }_{2n}\\ ⋮& ⋮& \ddots & ⋮\\ {\alpha }_{m1}& {\alpha }_{m2}& \dots & {\alpha }_{mn}\end{array}\right],定义:$$$$A_r^t=\left[\begin{array}{cccc}{\left({\alpha }_{11}\right)}_{rot}^T& {\left({\alpha }_{12}\right)}_{rot}^T& \dots & {\left({\alpha }_{1n}\right)}_{rot}^T\\ {\left({\alpha }_{21}\right)}_{rot}^T& {\left({\alpha }_{22}\right)}_{rot}^T& \dots & {\left({\alpha }_{2n}\right)}_{rot}^T\\ ⋮& ⋮& \ddots & ⋮\\ {\left({\alpha }_{m1}\right)}_{rot}^T& {\left({\alpha }_{m2}\right)}_{rot}^T& \dots & {\left({\alpha }_{mn}\right)}_{rot}^T\end{array}\right]$$

---

• 12.

$$对于四维数组$$$$A=\left[\begin{array}{cccc}{\alpha }_{11}& {\alpha }_{12}& \dots & {\alpha }_{1n}\\ {\alpha }_{21}& {\alpha }_{22}& \dots & {\alpha }_{2n}\\ ⋮& ⋮& \ddots & ⋮\\ {\alpha }_{m1}& {\alpha }_{m2}& \dots & {\alpha }_{mn}\end{array}\right],定义:$$$$A^T=\left[\begin{array}{cccc}{\alpha }_{11}^T& {\alpha }_{21}^T& \dots & {\alpha }_{m1}^T\\ {\alpha }_{12}^T& {\alpha }_{22}^T& \dots & {\alpha }_{m2}^T\\ ⋮& ⋮& \ddots & ⋮\\ {\alpha }_{1n}^T& {\alpha }_{2n}^T& \dots & {\alpha }_{mn}^T\end{array}\right]$$

---

• 13.

$$tr\left(A\cdot \left(B{\ast }_{valid}C\right)\right)=tr\left(\left(A{\ast }_{full}{B}_r^t\right)\cdot C\right)=tr\left(\left(C_r^t{\ast }_{valid}A\right)\cdot B\right)$$$$A\in {\mathbb{R}}^{n\times m\times k-l+1\times k-l+1},B\in {\mathbb{R}}^{m\times p\times l\times l},C\in {\mathbb{R}}^{p\times n\times k\times k},且k\ge l\ge 2.$$

---

• 14.

$$tr\left(A\cdot \left(B{\ast }_{full}C\right)\right)=tr\left(\left(A{\ast }_{valid}{B}_r^t\right)\cdot C\right)=tr\left(\left(C_r^t{\ast }_{valid}A\right)\cdot B\right)$$$$A\in {\mathbb{R}}^{n\times m\times k+l-1\times k+l-1},B\in {\mathbb{R}}^{m\times p\times l\times l},C\in {\mathbb{R}}^{p\times n\times k\times k},且k\ge 2,l\ge 2.$$

---

• 15.

$$A,B为四维数组,则:$$$$(A\ast B{)}^T=B^T\ast A^T$$

---