UTF8gbsn
关于矩阵向量的导数在优化问题,还有机器学习领域比较常遇到.在这里我们进行一次梳理,不过本文只介绍一阶函数的导数.
后面再逐渐写文来论证其他的导数.
-
∂ x T a ∂ x = ∂ a T x ∂ x = a \frac{\partial \mathbf{x}^T\mathbf{a}}{\partial \mathbf{x}}=\frac{\partial \mathbf{a}^T\mathbf{x}}{\partial \mathbf{x}}=\mathbf{a} ∂x∂xTa=∂x∂aTx=a
∂ ( a 1 x 1 + a 1 x 2 + ⋯ + a 1 x n ) ∂ x 1 = a 1 ∂ ( a 1 x 1 + a 1 x 2 + ⋯ + a 1 x n ) ∂ x 2 = a 2 ⋮ = ⋮ ∂ ( a 1 x 1 + a 1 x 2 + ⋯ + a 1 x n ) ∂ x n = a n \left. \begin{aligned} \frac{\partial \left( a_1x_1+ a_1x_2+ \cdots+ a_1x_n \right)}{\partial x_1}&=&a_1\\ \frac{\partial \left( a_1x_1+ a_1x_2+ \cdots+ a_1x_n \right)}{\partial x_2}&=&a_2\\ \vdots\quad\quad\quad\quad\quad&=&\vdots\\ \frac{\partial \left( a_1x_1+ a_1x_2+ \cdots+ a_1x_n \right)}{\partial x_n}&=&a_n \end{aligned} \right. ∂x1∂(a1x1+a1x2+⋯+a1xn)∂x2∂(a1x1+a1x2+⋯+a1xn)⋮∂xn∂(a1x1+a1x2+⋯+a1xn)====a1a2⋮an 由此可见原式得证. -
∂ a T X b ∂ X = a b T \frac{\partial \mathbf{a^TXb}}{\partial \mathbf{X}}=\mathbf{ab^T} ∂X∂aTXb=abT
-
a n T X n × m b m = ( a 1 , a 2 , ⋯ , a n ) ( x 11 x 12 ⋯ x 1 m x 21 x 22 ⋯ x 2 m ⋮ ⋮ ⋮ x n 1 x n 2 ⋯ x n m ) ( b 1 b 2 ⋮ b m ) \mathbf{a^T_nX_{n\times m}b_m}=\left( a_1, a_2, \cdots, a_n \right)\left( \begin{array}{cccc} x_{11} & x_{12} & \cdots & x_{1m} \\ x_{21} & x_{22} & \cdots & x_{2m} \\ \vdots & \vdots & & \vdots \\ x_{n1} & x_{n2} & \cdots & x_{nm} \end{array} \right)\left( \begin{array}{c} b_1 \\ b_2 \\ \vdots \\ b_m \end{array} \right) anTXn×mbm=(a1,a2,⋯,an)⎝⎜⎜⎜⎛x11x21⋮xn1x12x22⋮xn2⋯⋯⋯x1mx2m⋮xnm⎠⎟⎟⎟⎞⎝⎜⎜⎜⎛b1b2⋮bm⎠⎟⎟⎟⎞
-
∂ a T X b ∂ X = ( a 1 b 1 a 1 b 2 ⋯ a 1 b m a 2 b 1 a 2 b 2 ⋯ a 2 b m ⋮ ⋮ ⋮ a n b 1 a n b 2 ⋯ a n b m ) = a b T \frac{\partial \mathbf{a^TXb}}{\partial \mathbf{X}} = \left( \begin{array}{cccc} a_1b_1 & a_1b_2 & \cdots & a_1b_m \\ a_2b_1 & a_2b_2 & \cdots & a_2b_m \\ \vdots & \vdots & & \vdots \\ a_nb_1 & a_nb_2 & \cdots & a_nb_m \end{array} \right)=\mathbf{ab^T} ∂X∂aTXb=⎝⎜⎜⎜⎛a1b1a2b1⋮anb1a1b2a2b2⋮anb2⋯⋯⋯a1bma2bm⋮anbm⎠⎟⎟⎟⎞=abT
-
-
∂ a T X T b ∂ X = b a T \frac{\partial \mathbf{a}^{T} \mathbf{X}^{T} \mathbf{b}}{\partial \mathbf{X}}=\mathbf{b a}^{T} ∂X∂aTXTb=baT,证明同上.
-
∂ a T X a ∂ X = ∂ a T X T a ∂ X = a a T \frac{\partial \mathbf{a}^{T} \mathbf{X} \mathbf{a}}{\partial \mathbf{X}}=\frac{\partial \mathbf{a}^{T} \mathbf{X}^{T} \mathbf{a}}{\partial \mathbf{X}}=\mathbf{aa}^{T} ∂X∂aTXa=∂X∂aTXTa=aaT,
只是上式的特列. -
∂ X ∂ X i j = J i j \frac{\partial \mathbf{X}}{\partial X_{i j}}=\mathbf{J}^{i j} ∂Xij∂X=Jij,其中 J i j \mathbf{J}^{i j} Jij定义为
J 2 , 3 = ( 0 0 0 0 0 0 0 1 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 ) J^{2,3}=\left( \begin{array}{ccccc} 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 1 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 \\ 0 & 0 & 0 & 0 & 0 \end{array} \right) J2,3=⎝⎜⎜⎜⎜⎛0000000000010000000000000⎠⎟⎟⎟⎟⎞ -
∂ ( X A ) i j ∂ X m n = δ i m A n j = ( J m n A ) i j \frac{\partial(\mathbf{X} \mathbf{A})_{i j}}{\partial X_{m n}}=\delta_{i m}\mathbf{A}_{n j}=\left(\mathbf{J}^{m n} \mathbf{A}\right)_{i j} ∂Xmn∂(XA)ij=δimAnj=(JmnA)ij,
这个等式的证明很简单,
直接写出矩阵形式就可.其中 δ i m = 1 i f f m = i \delta_{im}=1\quad iff \quad m=i δim=1iffm=i -
∂ ( X T A ) i j ∂ X m n = δ i n ( A ) m j = ( J n m A ) i j \frac{\partial\left(\mathbf{X}^{T} \mathbf{A}\right)_{i j}}{\partial X_{m n}}=\delta_{i n}(\mathbf{A})_{m j}=\left(\mathbf{J}^{n m} \mathbf{A}\right)_{i j} ∂Xmn∂(XTA)ij=δin(A)mj=(JnmA)ij,
和上面的式子同理.
1532
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
