矩阵微分

Contents

• Notation
• Derivatives of Linear Products
• Derivatives of Quadratic Products

Notation

• d/dx (y) isa vector whose (i) elementis dy(i)/dx
• d/dx (y) is a vectorwhose (i) elementis dy/dx(i)
• d/dx (y^T)is a matrixwhose (i,j) elementis dy(j)/dx(i)
• d/dx (Y) is a matrixwhose (i,j) elementis dy(i,j)/dx
• d/dX (y) is a matrixwhose (i,j) elementis dy/dx(i,j)

Note that the Hermitian transpose is not used because complexconjugates are not analytic.

In the expressions below matrices andvectors A, B, C donot depend on X.

Derivatives of Linear Products

• d/dx (AYB) =A * d/dx (Y)* B
• • d/dx (Ay) =A * d/dx (y)
• d/dx (x^TA) =A
• • d/dx (x^T) =I
  • d/dx (x^Ta) =d/dx (a^Tx)= a
• d/dX (a^TXb)= ab^T
• • d/dX (a^TXa)= d/dX (a^TX^Ta)= aa^T
• d/dX (a^TX^Tb)= ba^T
• d/dx (YZ) =Y * d/dx (Z)+ d/dx (Y) *Z

Derivatives of Quadratic Products

• d/dx (Ax+b)^TC(Dx+e)= A^TC(Dx+e) + D^TC^T(Ax+b)
• • d/dx (x^TCx)= (C+C^T)x
  • • [C:symmetric]: d/dx (x^TCx)= 2Cx
    • d/dx (x^Tx)= 2x
  • d/dx (Ax+b)^T (Dx+e)= A^T (Dx+e) + D^T (Ax+b)
  • • d/dx (Ax+b)^T (Ax+b)= 2A^T (Ax+b)
  • [C:symmetric]: d/dx (Ax+b)^TC(Ax+b)= 2A^TC(Ax+b)
• d/dX (a^TX^TXb)= X(ab^T +ba^T)
• • d/dX (a^TX^TXa)= 2Xaa^T
• d/dX (a^TX^TCXb)= C^TXab^T +CXba^T
• • d/dX (a^TX^TCXa)= (C +C^T)Xaa^T
  • [C:Symmetric] d/dX (a^TX^TCXa)= 2CXaa^T
• d/dX ((Xa+b)^TC(Xa+b))=(C+C^T)(Xa+b)a^T

Derivatives of Cubic Products

• d/dx (x^TAxx^T)=(A+A^T)xx^T+x^TAxI

Derivatives of Inverses

• d/dx (Y^-1)= -Y^-1d/dx (Y)Y^-1

Derivative of Trace

Note: matrix dimensions must result inan n*n argument fortr().

• d/dX (tr(X))= I
• d/dX (tr(X^k))=k(X^k-1)^T
• d/dX (tr(AX^k))= SUM_r=0:k-1(X^rAX^k-r-1)^T
• d/dX (tr(AX^-1B))= -(X^-1BAX^-1)^T
  • d/dX (tr(AX^-1))=d/dX (tr(X^-1A))= -X^-TA^TX^-T
• d/dX (tr(A^TXB^T))= d/dX (tr(BX^TA))= AB
  • d/dX (tr(XA^T))= d/dX (tr(A^TX))=d/dX (tr(X^TA))= d/dX (tr(AX^T)) =A
• d/dX (tr(AXBX^T))= A^TXB^T + AXB
  • d/dX (tr(XAX^T))= X(A+A^T)
  • d/dX (tr(X^TAX))= X^T(A+A^T)
  • d/dX (tr(AX^TX))= (A+A^T)X
• d/dX (tr(AXBX))= A^TX^TB^T + B^TX^TA^T
• [C:symmetric] d/dX (tr((X^TCX)^-1A)= d/dX (tr(A(X^TCX)^-1)= -(CX(X^TCX)^-1)(A+A^T)(X^TCX)^-1
• [B,C:symmetric] d/dX (tr((X^TCX)^-1(X^TBX))= d/dX (tr((X^TBX)(X^TCX)^-1)= -2(CX(X^TCX)^-1)X^TBX(X^TCX)^-1 +2BX(X^TCX)^-1

Derivative of Determinant

Note: matrix dimensions must result inan n*n argument fordet().

• d/dX (det(X))= d/dX (det(X^T))= det(X)*X^-T
• • d/dX (det(AXB)) =det(AXB)*X^-T
  • d/dX (ln(det(AXB)))= X^-T
• d/dX (det(X^k))= k*det(X^k)*X^-T
  • d/dX (ln(det(X^k)))= kX^-T
• [Real] d/dX (det(X^TCX))=det(X^TCX)*(C+C^T)X(X^TCX)^-1
• • [C: Real,Symmetric] d/dX (det(X^TCX))= 2det(X^TCX)*CX(X^TCX)^-1
• [C: Real,Symmetricc] d/dX (ln(det(X^TCX)))= 2CX(X^TCX)^-1

Jacobian

If y is a functionof x,then dy^T/dx isthe Jacobian matrixof y with respectto x.

Its determinant,|dy^T/dx|, isthe Jacobian of y withrespect to x andrepresents the ratio of thehyper-volumes dy and dx.The Jacobian occurs when changing variables in an integration:Integral(f(y)dy)=Integral(f(y(x))|dy^T/dx|dx).

Hessian matrix

If f is a functionof x then the symmetricmatrixd²f/dx² = d/dx^T(df/dx)is the Hessian matrix off(x). A valueof x for whichdf/dx = 0 correspondsto a minimum, maximum or saddle point according to whether theHessian is positive definite, negative definite or indefinite.

• d²/dx² (a^Tx)= 0
• d²/dx² (Ax+b)^TC(Dx+e)= A^TCD + D^TC^TA
• • d²/dx² (x^TCx)= C+C^T
  • • d²/dx² (x^Tx)= 2I
  • d²/dx² (Ax+b)^T (Dx+e)= A^TD + D^TA
  • • d²/dx² (Ax+b)^T (Ax+b)= 2A^TA
  • [C:symmetric]: d²/dx² (Ax+b)^TC(Ax+b)= 2A^TCA  

http://www.psi.toronto.edu/matrix/calculus.html

http://www.stanford.edu/~dattorro/matrixcalc.pdf

http://www.colorado.edu/engineering/CAS/courses.d/IFEM.d/IFEM.AppD.d/IFEM.AppD.pdf

http://www4.ncsu.edu/~pfackler/MatCalc.pdf

http://center.uvt.nl/staff/magnus/wip12.pdf