矩阵微分(一)
标准梯度公式
自变量是标量
D f ( x ) = lim t → 0 f ( x + t ) − f ( x ) t Df(x) = \lim _{t\to 0} \frac {f(x+t)-f(x)}{t} Df(x)=t→0limtf(x+t)−f(x)
自变量是向量
D w f ( x ) = lim t → 0 f ( x + t w ) − f ( x ) t D_{\textbf {w}}f(\textbf {x}) = \lim _{t\to 0} \frac {f(\textbf {x} + t\textbf {w}) - f(\textbf {x})}{t} Dwf(x)=t→0limtf(x+tw)−f(x)
自变量是矩阵
D W f ( X ) = lim t → 0 f ( X + t W ) − f ( X ) t D_{\textbf {W}}f(\textbf {X}) = \lim _{t\to 0} \frac {f(\textbf {X}+t\textbf {W})-f(\textbf {X})}{t} DWf(X)=t→0limtf(X+tW)−f(X)
矩阵迹的性质
性质1
t r ( A ) = t r ( A T ) tr(A) = tr(A^{T}) tr(A)=tr(AT)
性质2
t r ( A B ) = t r ( B A ) tr(AB) = tr(BA) tr(AB)=tr(BA)
t r ( A B ) = t r ( B A ) tr(AB) = tr(BA) tr(AB)=tr(BA)
t r ( A B C D ) = t r ( D A B C ) = t r ( C D A B ) = t r ( B C D A ) tr(ABCD) = tr(DABC) = tr(CDAB) = tr(BCDA) tr(ABCD)=tr(DABC)=tr(CDAB)=tr(BCDA)
性质3
t r ( A + B ) = t r ( A ) + t r ( B ) tr(A+B) = tr(A) + tr(B) tr(A+B)=tr(A)+tr(B)
性质4
t r ( α A ) = α t r ( A ) tr(\alpha A) = \alpha tr(A) tr(αA)=αtr(A)
性质5
设有矩阵H、U,H和U都是n x m的矩阵,则有:
∑ j = 1 m ∑ i = 1 n ( h i j u i j ) = ∑ j = 1 m ∑ i = 1 n ( ( h T ) j i u i j ) = t r ( H T U ) \sum _{j=1}^{m} \sum _{i=1}^{n}(h_{ij}u_{ij}) = \sum _{j=1}^{m} \sum _{i=1}^{n}((h^{T})_{ji}u_{ij}) = tr(H^{T}U) j=1∑mi=1∑n(hijuij)=j=1∑mi=1∑n((hT)jiuij)=tr(HTU)
矩阵微分的性质
设有关于矩阵A的一个函数f,记为f(A),f(A)关于A的导数为:
∇ A f ( A ) = ∂ f ( A ) ∂ A = [ ∂ f ∂ A 11 ∂ f ∂ A 12 ⋯ ∂ f ∂ A 1 n ∂ f ∂ A 21 ∂ f ∂ A 22 ⋯ ∂ f ∂ A 2 n ⋮ ⋮ ⋱ ⋮ ∂ f ∂ A m 1 ∂ f ∂ A m 2 ⋯ ∂ f ∂ A m n ] \nabla _{A}f(A) = \frac { \partial f(A) }{ \partial A } = \left[ \begin{matrix} \frac {\partial f }{\partial A_{11}}&\frac {\partial f }{\partial A_{12}}&\cdots &\frac {\partial f }{\partial A_{1n}}\\ \frac {\partial f }{\partial A_{21}}&\frac {\partial f }{\partial A_{22}}&\cdots &\frac {\partial f }{\partial A_{2n}}\\ \vdots &\vdots &\ddots &\vdots \\ \frac {\partial f }{\partial A_{m1}}&\frac {\partial f }{\partial A_{m2}}&\cdots &\frac {\partial f }{\partial A_{mn}}\\ \end{matrix} \right] ∇Af(A)=∂A∂f(A)