导数、梯度、雅可比和海森矩阵

参考：Jacobian matrix and determinant

1. Derivative

Derivative can be defined on functions of a single variable ($f:\mathbb{R}\to \mathbb{R}$).

2. Gradient

Gradient is a multi-variable generalization of the derivative ($f:{\mathbb{R}}^n\to \mathbb{R}$).
While a derivative can be defined on functions of a single variable, for scalar functions of several variables, the gradient takes its place.
$$\nabla f(p)=\left[\begin{array}{c}\frac{\partial f}{\partial x_1}(p)\\ ⋮\\ \frac{\partial f}{\partial x_n}(p)\end{array}\right]$$

3. Jacobian

雅克比矩阵实际上是对于梯度的一种泛化，对于值为标量的多变量的函数 $f(x)$，我们使用梯度，对于值为向量的多变量的函数，使用雅可比矩阵.
The Jacobian is the generalization of the gradient for vector-valued functions of several variables($f:{\mathbb{R}}^n\to {\mathbb{R}}^m$).
J = [ ∂ f ∂ x 1 ⋯ ∂ f ∂ x n ] = [ ∇ T f 1 ⋮ ∇ T f m ] = [ ∂ f 1 ∂