转自:http://www.psi.toronto.edu/matrix/calculus.html
####Matrix Manual: Matrix Calculus
- Notation
- Derivatives of Linear Products
- Derivatives of Quadratic Products
Notation
- d/dx (y) is a vector whose (i) element is dy(i)/dx
- d/dx (y) is a vector whose (i) element is dy/dx(i)
- d/dx (yT) is a matrix whose (i,j) element is dy(j)/dx(i)
- d/dx (Y) is a matrix whose (i,j) element is dy(i,j)/dx
- d/dX (y) is a matrix whose (i,j) element is dy/dx(i,j)
#Note that the Hermitian transpose is not used because complex conjugates are not analytic.
#In the expressions below matrices and vectors A, B, C do not depend on X.
Derivatives of Linear Products
- d/dx (AYB)=A \* d/dx (Y) * B
- d/dx (Ay) =A * d/dx (y)
- d/dx (xTA) =A
- d/dx (xT) =I
- d/dx (xTa) = d/dx (aTx) = a
- d/dX (aTXb) = abT
- d/dX (aTXa) = d/dX (aTXTa) = aaT
- d/dX (aTXTb) = baT
- d/dx (YZ) =Y \* d/dx (Z) + d/dx (Y) \* Z
Derivatives of Quadratic Products
- d/dx (Ax+b)TC(Dx+e) = ATC(Dx+e) + DTCT(Ax+b)
- d/dx (xTCx) = (C+CT)x
- [C: symmetric]: d/dx (xTCx) = 2Cx
- d/dx (xTx) = 2x
- d/dx (Ax+b)T (Dx+e) = AT (Dx+e) + DT (Ax+b)
- d/dx (Ax+b)T (Ax+b) = 2AT (Ax+b)
- [C: symmetric]: d/dx (Ax+b)TC(Ax+b) = 2ATC(Ax+b)
- d/dX (aTXTXb) = X(abT + baT)
- d/dX (aTXTXa) = 2XaaT
- d/dX (aTXTCXb) = CTXabT + CXbaT
- d/dX (aTXTCXa) = (C + CT)XaaT
- [C:Symmetric] d/dX (aTXTCXa) = 2CXaaT
- d/dX ((Xa+b)TC(Xa+b)) = (C+CT)(Xa+b)aT
Derivatives of Cubic Products
- d/dx (xTAxxT) = (A+AT)xxT+xTAxI
Derivatives of Inverses
- d/dx (Y-1) = -Y-1d/dx (Y)Y-1
Derivative of Trace
Note: matrix dimensions must result in an n*n argument for tr().
- d/dX (tr(X)) = I
- d/dX (tr(Xk)) =k(Xk-1)T
- d/dX (tr(AXk)) = SUMr=0:k-1(XrAXk-r-1)T
- d/dX (tr(AX-1B)) = -(X-1BAX-1)T
- d/dX (tr(AX-1)) =d/dX (tr(X-1A)) = -X-TATX-T
- d/dX (tr(ATXBT)) = d/dX (tr(BXTA)) = AB
- d/dX (tr(XAT)) = d/dX (tr(ATX)) =d/dX (tr(XTA)) = d/dX (tr(AXT)) = A
- d/dX (tr(AXBXT)) = ATXBT + AXB
- d/dX (tr(XAXT)) = X(A+AT)
- d/dX (tr(XTAX)) = XT(A+AT)
- d/dX (tr(AXTX)) = (A+AT)X
- d/dX (tr(AXBX)) = ATXTBT + BTXTAT
- [C:symmetric] d/dX (tr((XTCX)-1A) = d/dX (tr(A (XTCX)-1) = -(CX(XTCX)-1)(A+AT)(XTCX)-1
- [B,C:symmetric] d/dX (tr((XTCX)-1(XTBX)) = d/dX (tr( (XTBX)(XTCX)-1) = -2(CX(XTCX)-1)XTBX(XTCX)-1 + 2BX(XTCX)-1