矩阵求导与Hessian矩阵

最新推荐文章于 2025-01-15 12:56:22 发布

韩明宇

最新推荐文章于 2025-01-15 12:56:22 发布

阅读量1.1w

点赞数 7

分类专栏：数学基础

本文链接：https://blog.csdn.net/qq_37098526/article/details/90720936

版权

数学基础专栏收录该内容

6 篇文章

订阅专栏

标量关于标量的导数

$\frac{dy}{dx}$

向量关于标量的导数

设向量 $y=\begin{bmatrix} y_{1}\\ y_{2}\\ ...\\ y_{n} \end{bmatrix}$ 和标量x，

则 $\frac{dy}{dx}=\begin{bmatrix} \frac{dy_{1}}{dx}\\ \frac{dy_{2}}{dx}\\ ...\\ \frac{dy_{n}}{dx} \end{bmatrix}$

矩阵关于标量的导数

设M×N矩阵 $Y=\begin{bmatrix} y_{11} &y_{12} &... &y_{1n} \\ y_{21} &y_{22} &... &y_{2n} \\ ... &... &... &... \\ y_{m1} &y_{m2} &... &y_{mn} \end{bmatrix}$ 和标量x，

则 $\frac{dY}{dx}=\begin{bmatrix} \frac{dy_{11}}{dx} &\frac{dy_{12}}{dx} &... &\frac{dy_{1n}}{dx} \\ \frac{dy_{21}}{dx} &\frac{dy_{22}}{dx} &... &\frac{dy_{2n}}{dx} \\ ... &... &... &... \\ \frac{dy_{m1}}{dx} &\frac{dy_{m2}}{dx} &... &\frac{dy_{mn}}{dx} \end{bmatrix}$

标量关于向量的导数

设标量y和向量 $x=\begin{bmatrix} x_{1}\\ x_{2}\\ ...\\ x_{n} \end{bmatrix}$ ，

则 $\frac{dy}{dx}=\begin{bmatrix} \frac{\partial y}{\partial x_{1}}\\ \frac{\partial y}{\partial x_{2}}\\ ...\\ \frac{\partial y}{\partial x_{n}} \end{bmatrix}$

向量关于向量的导数

设向量 $y=\begin{bmatrix} y_{1}\\ y_{2}\\ ...\\ y_{m} \end{bmatrix}$ 和向量 $x=\begin{bmatrix} x_{1}& x_{2}& ...& x_{n} \end{bmatrix}$ ，

则 $\frac{dy}{dx}=\begin{bmatrix} \frac{\partial y_{1}}{\partial x_{1}} &\frac{\partial y_{1}}{\partial x_{2}} &... &\frac{\partial y_{1}}{\partial x_{n}} \\ \frac{\partial y_{2}}{\partial x_{1}} &\frac{\partial y_{2}}{\partial x_{2}} &... &\frac{\partial y_{2}}{\partial x_{n}} \\ ... &... &... &... \\ \frac{\partial y_{m}}{\partial x_{1}} &\frac{\partial y_{m}}{\partial x_{2}} &... &\frac{\partial y_{m}}{\partial x_{n}} \end{bmatrix}$ ，即Jacobian矩阵。

矩阵关于向量的导数

设M×N矩阵 $Y=\begin{bmatrix} y_{11} &y_{12} &... &y_{1n} \\ y_{21} &y_{22} &... &y_{2n} \\ ... &... &... &... \\ y_{m1} &y_{m2} &... &y_{mn} \end{bmatrix}$ 和p维向量 $x=\begin{bmatrix} x_{1}\\ x_{2}\\ ...\\ x_{p} \end{bmatrix}$ ，

则 $\frac{dY}{dx}=\begin{bmatrix} \frac{\partial Y}{\partial x_{1}} \\ \frac{\partial Y}{\partial x_{2}} \\... \\ \frac{\partial Y}{\partial x_{p}} \end{bmatrix}$ ，其中 $\frac{\partial Y}{\partial x_{i}}=\begin{bmatrix} \frac{\partial y_{11}}{\partial x_{i}} &\frac{\partial y_{12}}{\partial x_{i}} &... &\frac{\partial y_{1n}}{\partial x_{i}} \\ \frac{\partial y_{21}}{\partial x_{i}} &\frac{\partial y_{22}}{\partial x_{i}} &... &\frac{\partial y_{2n}}{\partial x_{i}} \\ ... &... &... &... \\ \frac{\partial y_{m1}}{\partial x_{i}} &\frac{\partial y_{m2}}{\partial x_{i}} &... &\frac{\partial y_{mn}}{\partial x_{i}} \end{bmatrix}$

标量关于矩阵的导数

设标量y和M×N矩阵 $X=\begin{bmatrix} x_{11} &x_{12} &... &x_{1n} \\ x_{21} &x_{22} &... &x_{2n} \\ ... &... &... &... \\ x_{m1} &x_{m2} &... &x_{mn} \end{bmatrix}$ ，

向量关于矩阵的导数

设p维向量 $y=\begin{bmatrix} y_{1}\\ y_{2}\\ ...\\ y_{p} \end{bmatrix}$ 和M×N矩阵 $X=\begin{bmatrix} x_{11} &x_{12} &... &x_{1n} \\ x_{21} &x_{22} &... &x_{2n} \\ ... &... &... &... \\ x_{m1} &x_{m2} &... &x_{mn} \end{bmatrix}$ ，

则 $\frac{dy}{dX}=\begin{bmatrix} \frac{\partial y}{\partial x_{11}} &\frac{\partial y}{\partial x_{12}} &... &\frac{\partial y}{\partial x_{1n}} \\ \frac{\partial y}{\partial x_{21}} &\frac{\partial y}{\partial x_{22}} &... &\frac{\partial y}{\partial x_{2n}} \\ ... &... &... &... \\ \frac{\partial y}{\partial x_{m1}} &\frac{\partial y}{\partial x_{m2}} &... &\frac{\partial y}{\partial x_{mn}} \end{bmatrix}$ ，其中 $\frac{\partial y}{\partial x_{ij}}=\begin{bmatrix} \frac{\partial y_{1}}{\partial x_{ij}}\\ \frac{\partial y_{2}}{\partial x_{ij}}\\ ...\\ \frac{\partial y_{p}}{\partial x_{ij}} \end{bmatrix}$

矩阵关于矩阵的导数

设P×Q矩阵 $Y=\begin{bmatrix} y_{11} &y_{12} &... &y_{1q} \\ y_{21} &y_{22} &... &y_{2q} \\ ... &... &... &... \\ y_{p1} &y_{p2} &... &y_{pq} \end{bmatrix}$ 和M×N矩阵 $X=\begin{bmatrix} x_{11} &x_{12} &... &x_{1n} \\ x_{21} &x_{22} &... &x_{2n} \\ ... &... &... &... \\ x_{m1} &x_{m2} &... &x_{mn} \end{bmatrix}$ ，

则 $\frac{dY}{dX}=(\frac{\partial Y}{\partial x_{ij}})_{pm\times qn}=\begin{bmatrix} \frac{\partial Y}{\partial x_{11}} &\frac{\partial Y}{\partial x_{12}} &... &\frac{\partial Y}{\partial x_{1n}} \\ \frac{\partial Y}{\partial x_{21}} &\frac{\partial Y}{\partial x_{22}} &... &\frac{\partial Y}{\partial x_{2n}} \\ ... &... &... &... \\ \frac{\partial Y}{\partial x_{m1}} &\frac{\partial Y}{\partial x_{m2}} &... &\frac{\partial Y}{\partial x_{mn}} \end{bmatrix}$ ，

其中 $\frac{\partial Y}{\partial x_{ij}}=\begin{bmatrix} \frac{\partial y_{11}}{\partial x_{ij}} &\frac{\partial y_{12}}{\partial x_{ij}} &... &\frac{\partial y_{1q}}{\partial x_{ij}} \\ \frac{\partial y_{21}}{\partial x_{ij}} &\frac{\partial y_{22}}{\partial x_{ij}} &... &\frac{\partial y_{2q}}{\partial x_{ij}} \\ ... &... &... &... \\ \frac{\partial y_{p1}}{\partial x_{ij}} &\frac{\partial y_{p2}}{\partial x_{ij}} &... &\frac{\partial y_{pq}}{\partial x_{ij}} \end{bmatrix}$

Hessian矩阵

The above Hessian is of the the function $f:\mathbb{R}''\rightarrow \mathbb{R}$ where all second order partial derivatives of exist and are continuous throughout it's domain & the function is $f(x_{1},x_{2},...,x_{n})$