Gradient, Jacobian, Hessian, Laplacian and all about

In this article I will explain the different derivative operators used in calculus. Before we start looking into the operators let's first revise the different types of mathematical functions and the concept of derivatives.

In mathematics, a function is a mapping between a set of inputs and a set of permissible outputs with the property that each input is related to exactly one output.

If the mapping is from a scalar to another scalar then we call it a scalar function.

f:R→R

For example, the function f(x)=x2 is a mapping from the set of real numbers to the set of non-negative real numbers f:R→R

.

If the mapping is from a vector to a scalar then the function is known as a multivariable or multivariate function.

where n is the dimension of the input vector. For example, the function is a mapping from the set of 2-dimensional real vectors X=[x1,x2] to the set of real numbers f:R2→R

.

If the mapping is from a vector (or a scalar) to another vector then the function is known as a vector function or a vector-valued function.

f:Rn→Rm,

where n is the dimension of the input vector and m is the dimension of the output vector. For example, a function taking the 3-dimensional position as input and giving the velocity of the object in 3-dimensions is a vector valued function f:R3→R3

.

Derivative

Now let's look at what the derivative of a function is. The derivative of a function is a measure of how much the value of a function f(x)

changes when we change the input x to the function, change in f(x)change in x. In the case of scalar functions the concept of derivative is very simple as there is only one variable whose value need to be changed and there is only one output for which we need to measure the change. The derivative of a scalar function is denoted as

f′(x)=df(x)dx

For example, the derivative of position with respect to time tells us how much the position of an object changes when we advance time and this is known as the velocity. We can also repeat the derivative operator and find the derivative of the derivative, for example, the derivative of the velocity with respect to time is known as the accelaration, which is another very useful physical quantity.

Gradient

To generalize the notion of derivative to the multivariate functions we use the gradient operator. The gradient of a multivariate function is a vector with each component proportional to the derivative of the function with respect to that component. For example, if we have a three dimensional multivariate function, f(x1,x2,x3)

, then gradient is given by

 

Let's say the function f(x1,x2,x3) gives us the temperature of the point [x1,x2,x3] in a room. Now if we take a very small step in the x1 direction, and note the change in temperature, then we get the derivative of f with respect to direction x1,  , let its value be 5. Now if we repeat it in the other two directions, we will find out how much the temperature changes in x2 and x3 directions, let their values be 15 and 0, respectively. Since the value of the function changes 3 times more in the x2 direction than that of x1 direction, if we take a step in the [5,15,0] direction, it will give us maximum change in the temperature f(x1,x2,x3)

. Thus the gradient vector gives us the magnitude and direction of maximum change of a multivariate function.

Jacobian

The Jacobian operator is a generalization of the derivative operator to the vector-valued functions. As we have seen earlier, a vector-valued function is a mapping from 

, hence, now instead of having a scalar value of the function f, we will have a mapping . Thus, we now need the rate of change of each component of f with respect to each component of the input variable x, this is exactly what is captured by a matrix called Jacobian matrix 

Hessian

The gradient is the first order derivative of a multivariate function. To find the second order derivative of a multivariate function, we define a matrix called a Hessian matrix given by

Laplacian

The trace of the Hessian matrix is known as the Laplacian operator denoted by ∇2

,

I hope you enjoyed reading. Your feedback on this article will be highly appreciated.

 

thanks  Najeeb Khan  。source URL：https://najeebkhan.github.io/blog/VecCal.html