数值计算之梯度向量和梯度矩阵，雅可比矩阵，海森矩阵

最新推荐文章于 2024-07-04 10:26:47 发布

RuiH.AI

最新推荐文章于 2024-07-04 10:26:47 发布

阅读量7.9k

点赞数 5

分类专栏：概率论与机器学习文章标签：线性代数矩阵机器学习

本文链接：https://blog.csdn.net/qq_41035283/article/details/121631656

版权

概率论与机器学习专栏收录该内容

50 篇文章 87 订阅

订阅专栏

数值计算之梯度向量，雅可比矩阵，海森矩阵

前言
梯度向量
梯度矩阵
雅可比矩阵
海森矩阵
总结
补充

前言

非线性最小二乘中的函数求导内容，主要涉及梯度向量、雅可比矩阵和海森矩阵。因此提前做一个辨析。实际上之前在矩阵求导中已经提到过这些内容。

梯度向量

对于实值向量函数 $f(x)\in R,x=(x_1,x_2,\dots,x_n)^T$ ，其梯度向量可表示为：
$\nabla f(x)=\frac {\partial f(x)}{\partial x}= \begin{bmatrix} \frac {\partial f}{\partial x_1} \\ \frac {\partial f}{\partial x_2} \\ \dots \\ \frac {\partial f}{\partial x_n} \\ \end{bmatrix}$
梯度向量的布局与分母的布局相同，也就是分母布局。

梯度矩阵

对于实值矩阵函数 $f(X)\in R$ ，其梯度矩阵 $\nabla f(X)$ 可表示为：
$\nabla f(X)=\frac {\partial f^T(X)}{\partial X}= \begin{bmatrix} \frac {\partial f}{\partial x_{11}} & \frac {\partial f}{\partial x_{12}} & \dots & \frac {\partial f}{\partial x_{1n}} \\ \frac {\partial f}{\partial x_{21}} & \frac {\partial f}{\partial x_{22}} & \dots & \frac {\partial f}{\partial x_{2n}} \\ \dots & \dots & \dots & \dots\\ \frac {\partial f}{\partial x_{n1}} & \frac {\partial f}{\partial x_{n2}} & \dots & \frac {\partial f}{\partial x_{nn}} \\ \end{bmatrix}$
对于实值向量函数 $f_i(x)\in R,x=(x_1,x_2,\dots,x_n)^T$ 组成的向量 $(f_1(x),f_2(x),\dots,f_n(x))^T$ ，其梯度矩阵可表示为：
$\nabla f(x)=\frac {\partial f^T(x)}{\partial x} =\begin{bmatrix} \frac {\partial f_1}{\partial x_{1}} & \frac {\partial f_2}{\partial x_{1}} & \dots & \frac {\partial f_n}{\partial x_{1}} \\ \frac {\partial f_1}{\partial x_{2}} & \frac {\partial f_2}{\partial x_{2}} & \dots & \frac {\partial f_n}{\partial x_{2}} \\ \dots & \dots & \dots & \dots\\ \frac {\partial f_1}{\partial x_{n}} & \frac {\partial f_2}{\partial x_{n}} & \dots & \frac {\partial f_n}{\partial x_{n}} \\ \end{bmatrix}$
梯度矩阵需要保持分母的布局不变，也就是分母布局。

雅可比矩阵

对于实值矩阵函数 $f(X)\in R$ ，其雅可比矩阵 $J (X)$ 可表示为：
$J(X)=\frac {\partial f(X)}{\partial X^T}= \begin{bmatrix} \frac {\partial f}{\partial x_{11}} & \frac {\partial f}{\partial x_{21}} & \dots & \frac {\partial f}{\partial x_{n1}} \\ \frac {\partial f}{\partial x_{12}} & \frac {\partial f}{\partial x_{22}} & \dots & \frac {\partial f}{\partial x_{n2}} \\ \dots & \dots & \dots & \dots\\ \frac {\partial f}{\partial x_{1n}} & \frac {\partial f}{\partial x_{2n}} & \dots & \frac {\partial f}{\partial x_{nn}} \\ \end{bmatrix}$

对于由实值向量函数 $f_i(x)\in R,x=(x_1,x_2,\dots,x_n)^T$ 组成的向量 $(f_1(x),f_2(x),\dots,f_n(x))^T$ ，其雅可比矩阵可表示为：
$J(x)=\frac {\partial f(x)}{\partial x^T}=\begin{bmatrix} \frac {\partial f_1}{\partial x_1} & \frac {\partial f_1}{\partial x_2} & \dots & \frac {\partial f_1}{\partial x_n} \\ \frac {\partial f_2}{\partial x_1} & \frac {\partial f_2}{\partial x_2} & \dots & \frac {\partial f_2}{\partial x_n} \\ \dots & \dots & \dots & \dots \\ \frac {\partial f_n}{\partial x_1} & \frac {\partial f_n}{\partial x_2} & \dots & \frac {\partial f_n}{\partial x_n} \\ \end{bmatrix}$
雅可比矩阵需要保持分子的布局不变，也就是分子布局。

海森矩阵

对于实值向量函数 $f(x)\in R,x=(x_1,x_2,\dots,x_n)^T$ ，其海森矩阵 $H (x)$ 是函数的二阶导，实际上就是 $f (x)$ 的梯度向量对 $x$ 的雅可比矩阵：

$H(x)=J(\nabla f(x))=\frac{\partial \frac {\partial f(x)}{\partial x}}{\partial x^T} = \begin{bmatrix} \frac {\partial^2f}{\partial x_1\partial x_1} & \frac {\partial^2f}{\partial x_1\partial x_2} & \dots& \frac {\partial^2f}{\partial x_1\partial x_n} \\ \frac {\partial^2f}{\partial x_2\partial x_1} & \frac {\partial^2f}{\partial x_2\partial x_2} & \dots& \frac {\partial^2f}{\partial x_2\partial x_n} \\ \dots & \dots & \dots & \dots \\ \frac {\partial^2f}{\partial x_n\partial x_1} & \frac {\partial^2f}{\partial x_n\partial x_2} & \dots& \frac {\partial^2f}{\partial x_n\partial x_n} \\ \end{bmatrix}$