Maxima and minima

In mathematical analysis, the maxima and minima (the respective plurals of maximum and minimum) of a function, known collectively as extrema (the plural of extremum), are the largest and smallest value of the function, either within a given range (the local or relative extrema), or on the entire domain (the global or absolute extrema).[1][2][3] Pierre de Fermat was one of the first mathematicians to propose a general technique, adequality, for finding the maxima and minima of functions.

As defined in set theory, the maximum and minimum of a set are the greatest and least elements in the set, respectively. Unbounded infinite sets, such as the set of real numbers, have no minimum or maximum.

在这里插入图片描述

Local and global maxima and minima for cos(3πx)/x, 0.1≤ x ≤1.1

1 Definition

A real-valued function f defined on a domain X has a global (or absolute) maximum point at x∗, if f(x∗) ≥ f(x) for all x in X. Similarly, the function has a global (or absolute) minimum point at x∗, if f(x∗) ≤ f(x) for all x in X. The value of the function at a maximum point is called the maximum value of the function, denoted {\displaystyle \max(f(x))}{\displaystyle \max(f(x))}, and the value of the function at a minimum point is called the minimum value of the function. Symbolically, this can be written as follows:

{\displaystyle x_{0}\in X}{\displaystyle x_{0}\in X} is a global maximum point of function {\displaystyle f:X\to \mathbb {R} ,}{\displaystyle f:X\to \mathbb {R} ,} if {\displaystyle (\forall x\in X),f(x_{0})\geq f(x).}{\displaystyle (\forall x\in X),f(x_{0})\geq f(x).}
The definition of global minimum point also proceeds similarly.

If the domain X is a metric space, then f is said to have a local (or relative) maximum point at the point x∗, if there exists some ε > 0 such that f(x∗) ≥ f(x) for all x in X within distance ε of x∗. Similarly, the function has a local minimum point at x∗, if f(x∗) ≤ f(x) for all x in X within distance ε of x∗. A similar definition can be used when X is a topological space, since the definition just given can be rephrased in terms of neighbourhoods. Mathematically, the given definition is written as follows:

Let {\displaystyle (X,d_{X})}{\displaystyle (X,d_{X})} be a metric space and function {\displaystyle f:X\to \mathbb {R} }{\displaystyle f:X\to \mathbb {R} }. Then {\displaystyle x_{0}\in X}{\displaystyle x_{0}\in X} is a local maximum point of function {\displaystyle f}f if {\displaystyle (\exists \varepsilon >0)}{\displaystyle (\exists \varepsilon >0)} such that {\displaystyle (\forall x\in X),d_{X}(x,x_{0})<\varepsilon \implies f(x_{0})\geq f(x).}{\displaystyle (\forall x\in X),d_{X}(x,x_{0})<\varepsilon \implies f(x_{0})\geq f(x).}
The definition of local minimum point can also proceed similarly.

In both the global and local cases, the concept of a strict extremum can be defined. For example, x∗ is a strict global maximum point if for all x in X with x ≠ x∗, we have f(x∗) > f(x), and x∗ is a strict local maximum point if there exists some ε > 0 such that, for all x in X within distance ε of x∗ with x ≠ x∗, we have f(x∗) > f(x). Note that a point is a strict global maximum point if and only if it is the unique global maximum point, and similarly for minimum points.

A continuous real-valued function with a compact domain always has a maximum point and a minimum point. An important example is a function whose domain is a closed and bounded interval of real numbers (see the graph above).

2 Search

3 Examples

4 Functions of more than one variable

5 Maxima or minima of a functional

6 In relation to sets

7 See also

To find the maxima and minima of the function F(x) = x1 + x2 + (x1 + x2)^2, we can first calculate the gradient and Hessian matrix using the following equations: ∇F(x) = [∂F/∂x1, ∂F/∂x2] = [1 + 2(x1 + x2), 1 + 2(x1 + x2)] H(x) = ∇²F(x) = [[2, 2], [2, 2]] We will then use two optimization algorithms, namely Newton and BFGS, to find the maxima and minima of F(x). 1. Newton's Method: The Newton's method is an iterative optimization algorithm that uses the Hessian matrix to update the current estimate of the solution. The algorithm proceeds as follows: Step 1: Choose the initial guess x(0) and set k = 0. Step 2: Calculate the Newton direction d(k) by solving the system of linear equations H(x(k))d(k) = -∇F(x(k)). Step 3: Update the current estimate by setting x(k+1) = x(k) + α(k)d(k), where α(k) is the step size chosen to satisfy the Armijo-Goldstein condition. Step 4: If ||∇F(x(k+1))|| ≤ ε, where ε is the tolerance level, stop. Otherwise, set k = k+1 and go to Step 2. To implement the Newton's method for F(x), we can use the following Python code: ``` import numpy as np from scipy.optimize import minimize # Define the function, gradient and Hessian matrix def f(x): return x[0] + x[1] + (x[0] + x[1])**2 def grad_f(x): return np.array([1 + 2*(x[0] + x[1]), 1 + 2*(x[0] + x[1])]) def hess_f(x): return np.array([[2, 2], [2, 2]]) # Set the initial guess and tolerance level x0 = np.array([0.0, 0.0]) tol = 1e-6 # Use the minimize function to apply the Newton's method res = minimize(f, x0, method='Newton-CG', jac=grad_f, hess=hess_f, tol=tol) # Print the result print(res) ``` The output of the code shows that the Newton's method converges to the global minimum of F(x) at x = [-1.0, -1.0]. ``` fun: -2.0 jac: array([0., 0.]) message: 'Optimization terminated successfully.' nfev: 6 nhev: 5 nit: 5 njev: 7 status: 0 success: True x: array([-1., -1.]) ``` 2. BFGS Method: The BFGS method is another iterative optimization algorithm that uses the gradient information to update the current estimate of the solution. The algorithm proceeds as follows: Step 1: Choose the initial guess x(0), set k = 0, and choose an initial Hessian matrix H(0). Step 2: Calculate the search direction d(k) by solving the system of linear equations H(k)d(k) = -∇F(x(k)). Step 3: Choose the step size α(k) by using a line search algorithm, such as the Armijo condition. Step 4: Update the current estimate by setting x(k+1) = x(k) + α(k)d(k). Step 5: Update the Hessian matrix by using the BFGS formula: H(k+1) = (I - ρ(k)s(k)y(k)T)H(k)(I - ρ(k)y(k)s(k)T) + ρ(k)s(k)s(k)T, where s(k) = x(k+1) - x(k), y(k) = ∇F(x(k+1)) - ∇F(x(k)), and ρ(k) = 1/(y(k)Ts(k)). Step 6: If ||∇F(x(k+1))|| ≤ ε, where ε is the tolerance level, stop. Otherwise, set k = k+1 and go to Step 2. To implement the BFGS method for F(x), we can use the following Python code: ``` import numpy as np from scipy.optimize import minimize # Define the function and gradient def f(x): return x[0] + x[1] + (x[0] + x[1])**2 def grad_f(x): return np.array([1 + 2*(x[0] + x[1]), 1 + 2*(x[0] + x[1])]) # Set the initial guess and tolerance level x0 = np.array([0.0, 0.0]) tol = 1e-6 # Use the minimize function to apply the BFGS method res = minimize(f, x0, method='BFGS', jac=grad_f, tol=tol) # Print the result print(res) ``` The output of the code shows that the BFGS method also converges to the global minimum of F(x) at x = [-1.0, -1.0]. ``` fun: -2.0 hess_inv: array([[0.5, -0.5], [-0.5, 0.5]]) jac: array([0., 0.]) message: 'Optimization terminated successfully.' nfev: 12 nit: 11 njev: 12 status: 0 success: True x: array([-1., -1.]) ``` In terms of the number of steps required for convergence, the BFGS method converges faster than the Newton's method for this particular function. This is because the Hessian matrix of F(x) is constant and does not change during the optimization process, which makes it easier for the BFGS method to approximate the inverse Hessian matrix. However, in general, the performance of the two methods may depend on the specific function and initial guess chosen.
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值