矩阵计算规则

转自：http://www.psi.toronto.edu/matrix/calculus.html

####Matrix Manual: Matrix Calculus

• Notation
• Derivatives of Linear Products
• Derivatives of Quadratic Products

• Notation

• d/dx (y) is a vector whose (i) element is dy(i)/dx
• d/dx (y) is a vector whose (i) element is dy/dx(i)
• d/dx (y^T) is a matrix whose (i,j) element is dy(j)/dx(i)
• d/dx (Y) is a matrix whose (i,j) element is dy(i,j)/dx
• d/dX (y) is a matrix whose (i,j) element is dy/dx(i,j)

• #Note that the Hermitian transpose is not used because complex conjugates are not analytic.

  #In the expressions below matrices and vectors A, B, C do not depend on X.

  Derivatives of Linear Products

• d/dx (AYB)=A \* d/dx (Y) * B
• d/dx (Ay) =A * d/dx (y)
• d/dx (x^TA) =A
• d/dx (x^T) =I
• d/dx (x^Ta) = d/dx (a^Tx) = a
• d/dX (a^TXb) = ab^T
• d/dX (a^TXa) = d/dX (a^TX^Ta) = aa^T
• d/dX (a^TX^Tb) = ba^T
• d/dx (YZ) =Y \* d/dx (Z) + d/dx (Y) \* Z

• Derivatives of Quadratic Products

• d/dx (Ax+b)^TC(Dx+e) = A^TC(Dx+e) + D^TC^T(Ax+b)
• d/dx (x^TCx) = (C+C^T)x
• [C: symmetric]: d/dx (x^TCx) = 2Cx
• d/dx (x^Tx) = 2x
• d/dx (Ax+b)^T (Dx+e) = A^T (Dx+e) + D^T (Ax+b)
• d/dx (Ax+b)^T (Ax+b) = 2A^T (Ax+b)
• [C: symmetric]: d/dx (Ax+b)^TC(Ax+b) = 2A^TC(Ax+b)
• d/dX (a^TX^TXb) = X(ab^T + ba^T)
• d/dX (a^TX^TXa) = 2Xaa^T
• d/dX (a^TX^TCXb) = C^TXab^T + CXba^T
• d/dX (a^TX^TCXa) = (C + C^T)Xaa^T
• [C:Symmetric] d/dX (a^TX^TCXa) = 2CXaa^T
• d/dX ((Xa+b)^TC(Xa+b)) = (C+C^T)(Xa+b)a^T

• Derivatives of Cubic Products

• d/dx (x^TAxx^T) = (A+A^T)xx^T+x^TAxI

• Derivatives of Inverses

• d/dx (Y^-1) = -Y^-1d/dx (Y)Y^-1

• Derivative of Trace

  Note: matrix dimensions must result in an n*n argument for tr().

• d/dX (tr(X)) = I
• d/dX (tr(X^k)) =k(X^k-1)^T
• d/dX (tr(AX^k)) = SUM_r=0:k-1(X^rAX^k-r-1)^T
• d/dX (tr(AX^-1B)) = -(X^-1BAX^-1)^T
• d/dX (tr(AX^-1)) =d/dX (tr(X^-1A)) = -X^-TA^TX^-T
• d/dX (tr(A^TXB^T)) = d/dX (tr(BX^TA)) = AB
• d/dX (tr(XA^T)) = d/dX (tr(A^TX)) =d/dX (tr(X^TA)) = d/dX (tr(AX^T)) = A
• d/dX (tr(AXBX^T)) = A^TXB^T + AXB
• d/dX (tr(XAX^T)) = X(A+A^T)
• d/dX (tr(X^TAX)) = X^T(A+A^T)
• d/dX (tr(AX^TX)) = (A+A^T)X
• d/dX (tr(AXBX)) = A^TX^TB^T + B^TX^TA^T
• [C:symmetric] d/dX (tr((X^TCX)^-1A) = d/dX (tr(A (X^TCX)^-1) = -(CX(X^TCX)^-1)(A+A^T)(X^TCX)^-1
• [B,C:symmetric] d/dX (tr((X^TCX)^-1(X^TBX)) = d/dX (tr( (X^TBX)(X^TCX)^-1) = -2(CX(X^TCX)^-1)X^TBX(X^TCX)^-1 + 2BX(X^TCX)^-1