Introduction to nonlinear optimization第四章习题

(抄了个题，只做了以前上课写的题目，哪天有空了做一下）
4.1. Let $f\in C_L^{1,1}\left({\mathbb{R}}^n\right)$ and let { x k } k ≥ 0 \left\{\mathbf{x}_{k}\right\}_{k\ge 0} {xk​}k≥0​ be the sequence generated by the gradient method with a constant stepsize $t_k=\frac{1}{L}$.Assume that ${\mathbf{x}}_k\to {\mathbf{x}}^{\ast }$.Show that if $\nabla f\left({\mathbf{x}}_k\right)\ne 0$ for all $k\ge 0$,then ${\mathbf{x}}^{\ast }$ is not a local maximum point.

4.2. [9, Exercise 1.3.3] Consider the minimization problem
$$\min \left\{{\mathbf{x}}^T\mathbf{Q}\mathbf{x}:\mathbf{x}\in {\mathbb{R}}^2\right\}$$
where $\mathbf{Q}$ is a positive definite $2\times 2$ matrix. Suppose we use the diagonal scaling matrix
$$\mathbf{D}=\left(\begin{array}{cc}{\mathbf{Q}}_{11}^{-1}& 0\\ 0& {\mathbf{Q}}_{22}^{-1}\end{array}\right)$$
Show that the above scaling matrix improves the condition number of $\mathbf{Q}$ in the sense that
$$\chi \left({\mathbf{D}}^{\frac{1}{2}}\mathbf{Q}{\mathbf{D}}^{\frac{1}{2}}\right)\le \chi \left(\mathbf{Q}\right)$$

解：
设
$$\mathbf{Q}=\left(\begin{array}{cc}{Q}_{11}& Q_{12}\\ Q_{12}& Q_{22}\end{array}\right)$$
$${\mathbf{D}}^{\frac{1}{2}}\mathbf{Q}{\mathbf{D}}^{\frac{1}{2}}=\left(\begin{array}{cc}1& \frac{Q_{12}}{\sqrt{Q_{11}{Q}_{22}}}\\ \frac{Q_{12}}{\sqrt{Q_{11}{Q}_{22}}}& 1\end{array}\right)$$
因为$Q\succ 0$,所以
$$Q_{11},Q_{22}>0,Q_{11}{Q}_{22}>Q_{12}^2$$
因为
$$k\mathbf{Q}\succ 0,\left(k>0\right)$$
并且
$$\chi \left(\mathbf{Q}\right)=\chi \left(k\mathbf{Q}\right),\left(k>0\right)$$
考虑
$$\mathbf{A}=\left(\begin{array}{cc}\alpha & 1\\ 1& \beta \end{array}\right),\alpha ,\beta >0,\alpha \beta >1$$
可以手算出$\mathbf{A}$的特征值，于是
$$\chi \left(\mathbf{A}\right)=\frac{\alpha +\beta +\sqrt{{\left(\alpha -\beta \right)}^2+4}}{\alpha +\beta -\sqrt{{\left(\alpha -\beta \right)}^2+4}}=\frac{1+\sqrt{\frac{{\left(\alpha -\beta \right)}^2+4}{{\left(\alpha +\beta \right)}^2}}}{1-\sqrt{\frac{{\left(\alpha -\beta \right)}^2+4}{{\left(\alpha +\beta \right)}^2}}}$$

令$\mathbf{C}=\frac{1}{Q_{12}}\mathbf{Q}=\left(\begin{array}{cc}\alpha & 1\\ 1& \beta \end{array}\right)$
其中$\alpha =\frac{Q_{11}}{Q_{12}},\beta =\frac{Q_{22}}{Q_{12}}$
于是$\mathbf{P}=\frac{\sqrt{\alpha \beta }}{Q_{12}}{\mathbf{D}}^{\frac{1}{2}}\mathbf{Q}{\mathbf{D}}^{\frac{1}{2}}=\left(\begin{array}{cc}\sqrt{\alpha \beta }& 1\\ 1& \sqrt{\alpha \beta }\end{array}\right)$
于是
$$\chi \left(\mathbf{C}\right)=\chi \left(\mathbf{Q}\right)=\frac{1+\sqrt{\frac{{\left(\alpha -\beta \right)}^2+4}{{\left(\alpha +\beta \right)}^2}}}{1-\sqrt{\frac{{\left(\alpha -\beta \right)}^2+4}{{\left(\alpha +\beta \right)}^2}}}$$

$$\chi \left(\mathbf{P}\right)=\chi \left({\mathbf{D}}^{\frac{1}{2}}\mathbf{Q}{\mathbf{D}}^{\frac{1}{2}}\right)=\frac{1+\sqrt{\frac{1}{\alpha \beta }}}{1-\sqrt{\frac{1}{\alpha \beta }}}$$
$f(t)=\frac{1+\sqrt{t}}{1-\sqrt{t}}$在$\left(0,1\right)$上单调递增
因为$\frac{{\left(\alpha -\beta \right)}^2+4}{{\left(\alpha +\beta \right)}^2}>\frac{1}{\alpha \beta }$
所以$\chi \left(\mathbf{C}\right)>\chi \left(\mathbf{P}\right)$
所以
$$\chi \left({\mathbf{D}}^{\frac{1}{2}}\mathbf{Q}{\mathbf{D}}^{\frac{1}{2}}\right)\le \chi \left(\mathbf{Q}\right)$$

4.3. Consider the quadratic minimization problem
$$\min \left\{{\mathbf{x}}^T\mathbf{A}\mathbf{x}:\mathbf{x}\in {\mathbb{R}}^5\right\}$$
where $\mathbf{A}$ is the $5\times 5$ Hillbert matrix defined by
$${\mathbf{A}}_{i,j}=\frac{1}{i+j-1},i,j=1,2,3,4,5$$
The matrix can he constructed via the MATLAB command

A=hilb(5)

Run the following methods and compare the number of iterations required by each of the methods when the initial vector is ${\mathbf{x}}_0={\left(1,2,3,4,5\right)}^T$ to obtain a solution $\mathbf{x}$ with $\Vert \nabla f\left(\mathbf{x}\right)\Vert \le 10^{-4}$:

• gradient method with backtracking stepsize rule and parameters $\alpha =0.5,\beta =0.5,s=1$;
• gradient method with backtracking stepsize rule and parameters $\alpha =0.1,\beta =0.5,s=1$;
• gradient method with exact line search;
• diagonally scaled gradient method with diagonal elements $D_{ii}=\frac{1}{{\mathbf{A}}_{ii}},i=1,2,3,4,5$ and exact line search;
• diagonally scaled gradient method with diagonal elements $D_{ii}=\frac{1}{{\mathbf{A}}_{ii}},i=1,2,3,4,5$ and backtracking line search with parameters $\alpha =0.1,\beta =0.5,s=1$.

解：

function [x,fun_val]=gradient_method_backtracking(f,g,x0,s,alpha,...
    beta,epsilon)
% Gradient method with backtracking stepsize rule
%
% INPUT
%=======================================
% f ......... objective function
% g ......... gradient of the objective function
% x0......... initial point
% s ......... initial choice of stepsize
% alpha ..... tolerance parameter for the stepsize selection
% beta ...... the constant in which the stepsize is multiplied
%             at each backtracking step (0<beta<1)
% epsilon ... tolerance parameter for stopping rule
% OUTPUT
%=======================================
% x ......... optimal solution (up to a tolerance)
%             of min f(x)
% fun_val ... optimal function value
x=x0;
grad=g(x);
fun_val=f(x);
iter=0;
while (norm(grad)>epsilon)
    iter=iter+1;
    t=s;
    while (fun_val-f(x-t*grad)<alpha*t*norm(grad)^2)
        t=beta*t;
    end
    x=x-t*grad;
    fun_val=f(x);
    grad=g(x);
    fprintf('iter_number = %3d norm_grad = %2.6f fun_val = %2.6f \n',...
        iter,norm(grad),fun_val);
end

function [x,fun_val]=gradient_method_quadratic(A,b,x0,epsilon)
% INPUT
% ======================·
% A ....... the positive definite matrix associated with the
%           objective function
% b ....... a column vector associated with the linear part of the
%           objective function
% x0 ...... starting point of the method
%           epsilon . tolerance parameter
% OUTPUT
% =======================
% x ....... an optimal solution (up to a tolerance) of
%           min(x^T A x+2 b^T x)
% fun_val . the optimal function value up to a tolerance

x=x0;
iter=0;
grad=2*(A*x+b);
while (norm(grad)>epsilon)
     iter=iter+1;
     t=norm(grad)^2/(2*grad'*A*grad);
     x=x-t*grad;
     grad=2*(A*x+b);
     fun_val=x'*A*x+2*b'*x;
     fprintf('iter_number = %3d norm_grad = %2.6f fun_val = %2.6f\n',...
         iter,norm(grad),fun_val);
end

function [x,fun_val]=gradient_scaled_quadratic(A,b,D,x0,epsilon)
% INPUT
% ======================
% A ....... the positive definite matrix associated 
%           with the objective function
% b ....... a column vector associated with the linear part
%           of the objective function
% D ....... scaling matrix
% x0 ...... starting point of the method
% epsilon . tolerance parameter
% OUTPUT
% =======================
% x ....... an optimal solution (up to a tolerance) ...
%           of min(x^T A x+2 b^T x)
% fun_val . the optimal function value up to a tolerance

x=x0;
iter=0;
grad=2*(A*x+b);
while (norm(grad)>epsilon)
    iter=iter+1;
    t=grad'*D*grad/(2*(grad'*D')*A*(D*grad));
    x=x-t*D*grad;
    grad=2*(A*x+b);
    fun_val=x'*A*x+2*b'*x;
    fprintf('iter_number = %3d norm_grad = %2.6f fun_val = %2.6f \n',...
        iter,norm(grad),fun_val);
end

function [x,fun_val]=gradient_scaled_quadratic_backtracking(A,b,D,x0,s,...
    alpha,beta,epsilon)
% INPUT
% ======================
% A ....... the positive definite matrix associated 
%           with the objective function
% b ....... a column vector associated with the linear part
%           of the objective function
% D ....... scaling matrix
% x0....... initial point
% s ....... initial choice of stepsize
% alpha ... tolerance parameter for the stepsize selection
% beta .... the constant in which the stepsize is multiplied
%           at each backtracking step (0<beta<1)
% epsilon ... tolerance parameter for stopping rule
% OUTPUT
% =======================
% x ....... an optimal solution (up to a tolerance) ...
%           of min(x^T A x+2 b^T x)
% fun_val . the optimal function value up to a tolerance

x=x0;
iter=0;
grad=2*(A*x+b);
fun_val=x'*A*x+2*b'*x;
while (norm(grad)>epsilon)
    iter=iter+1;
    t=s;
    while (fun_val-((x-t*D*grad)'*A*(x-t*D*grad)+2*b'*(x-t*D*grad))<alpha*t*grad'*D*grad)
        t=beta*t;
    end
    x=x-t*D*grad;
    grad=2*(A*x+b);
    fun_val=x'*A*x+2*b'*x;
    fprintf('iter_number = %3d norm_grad = %2.6f fun_val = %2.6f \n',...
        iter,norm(grad),fun_val);
end

A=hilb(5);
b=zeros(size(A,2),1);
D=diag(1./diag(A));
f=@(x)x'*A*x;
g=@(x)2*A*x;
s=1;
alpha=0.5;
beta=0.5;
epsilon=1e-4;
x0=[1,2,3,4,5]';

s=1;
alpha=0.5;
beta=0.5;
gradient_method_backtracking(f,g,x0,s,alpha,beta,epsilon);

s=1;
alpha=0.1;
beta=0.5;
gradient_method_backtracking(f,g,x0,s,alpha,beta,epsilon);

gradient_method_quadratic(A,b,x0,epsilon);

s=1;
alpha=0.1;
beta=0.5;
gradient_scaled_quadratic_backtracking(A,b,D,x0,s,alpha,beta,epsilon);

回溯法：$\alpha =0.5,\beta =0.5,s=1$需要3301
回溯法：$\alpha =0.1,\beta =0.5,s=1$需要3732
精确线搜索：需要1271
diagonally scaled+精确线搜索：需要235
diagonally scaled+回溯法：需要104

4.4. Consider the Fermat-Weber problem
$$\underset{\mathbf{x}\in {\mathbb{R}}^n}{\min }\left\{f\left(\mathbf{x}\right)=\sum _{i=1}^m{\omega }_i\Vert \mathbf{x}-{\mathbf{a}}_i\Vert \right\}$$
where ${\omega }_1,\cdots ,{\omega }_m>0$ and ${\mathbf{a}}_1,\cdots ,{\mathbf{a}}_m\in {\mathbb{R}}^n$ are m different points. Let
$$p\in {\mathrm{argmin}}_{i=1,2,\cdots ,m}f\left({\mathbf{a}}_i\right)$$
Suppose that
$$\Vert \sum _{i\ne p}{\omega }_i\frac{{\mathbf{a}}_p-{\mathbf{a}}_i}{\Vert {\mathbf{a}}_p-{\mathbf{a}}_i\Vert }\Vert >{\omega }_p$$
(i)Show that there exists a direction $\mathbf{d}\in {\mathbb{R}}^n$ such that $f^{\prime }\left({\mathbf{a}}_p;d\right)<0$
(ii)Show that there exists ${\mathbf{x}}_0\in {\mathbb{R}}^n$ satisfying f ( x 0 ) < min ⁡ { f ( a 1 ) , ⋯   , f ( a p ) } f\left(\mathbf{x}_0\right)<\min\left\{f\left(\mathbf{a}_1\right),\cdots,f\left(\mathbf{a}_p\right)\right\} f(x0​)<min{f(a1​),⋯,f(ap​)}.Explain how to compute such a vector.

解：
(i)
$$\begin{array}{rl}{f}^{\prime }\left({\mathbf{a}}_p;d\right)& =\underset{t\to 0^{+}}{\lim }\frac{f\left({\mathbf{a}}_p+t\mathbf{d}\right)-f\left({\mathbf{a}}_p\right)}{t}\\ & =\underset{t\to 0^{+}}{\lim }\frac{{\sum }_{i\ne p}{\omega }_i\left(\Vert {\mathbf{a}}_p+t\mathbf{d}-{\mathbf{a}}_i\Vert -\Vert {\mathbf{a}}_p-{\mathbf{a}}_i\Vert \right)+{\omega }_p\Vert t\mathbf{d}\Vert }{t}\\ & \le \underset{t\to 0^{+}}{\lim }\frac{{\sum }_{i=1}^m{\omega }_i\Vert t\mathbf{d}\Vert }{t}=0\end{array}$$
然后不会了
(ii)

4.5. In the “source localization problem” we are given $m$ locations of sensors ${\mathbf{a}}_1,\cdots ,{\mathbf{a}}_m\in {\mathbb{R}}^n$ and approximate distances between the sensors and an unknown “source” located at $\mathbf{x}\in {\mathbb{R}}^n$:
$${\mathbf{d}}_i\approx \Vert \mathbf{x}-{\mathbf{a}}_i\Vert$$
The problem is to find and estimate $\mathbf{x}$ given the locations ${\mathbf{a}}_1,\cdots ,{\mathbf{a}}_m$ and the approximate distances $d_1,\cdots ,d_m$.A natural formulation as an optimization problem is to consider the nonlinear least squares problem
$$\text{ (SL) }\min \left\{f(\mathbf{x})\equiv \sum _{i=1}^m{\left(\Vert \mathbf{x}-{\mathbf{a}}_i\Vert -d_i\right)}^2\right\}$$
We will denote the set of sensors of A ≡ { a 1 , ⋯   , a m } \mathscr{A}\equiv \left\{\mathbf{a}_1,\cdots,\mathbf{a}_m\right\} A≡{a1​,⋯,am​}
(i)Show that the optimality condition $\nabla f\left(\mathbf{x}\right)=0\left(\mathbf{x}\notin \mathcal{A}\right)$ is the same as
$$\mathbf{x}=\frac{1}{m}\left\{\sum _{i=1}^m{\mathbf{a}}_i+\sum _{i=1}^m{d}_i\frac{\mathbf{x}-{\mathbf{a}}_i}{\Vert \mathbf{x}-{\mathbf{a}}_i\Vert }\right\}$$
(ii)Show that the corresponding fixed point method
$${\mathbf{x}}_{k+1}=\frac{1}{m}\left\{\sum _{i=1}^m{\mathbf{a}}_i+\sum _{i=1}^m{d}_i\frac{{\mathbf{x}}_k-{\mathbf{a}}_i}{\Vert {\mathbf{x}}_k-{\mathbf{a}}_i\Vert }\right\}$$
is a gradient method, assuming that ${\mathbf{x}}_k\notin \mathcal{A}$ for all $k\ge 0$.What is the stepsize?

解：
(i)
$$\nabla f\left(\mathbf{x}\right)=\sum _{i=1}^m\frac{\left(\Vert \mathbf{x}-{\mathbf{a}}_i\Vert -d_i\right)\left(\mathbf{x}-{\mathbf{a}}_i\right)}{\Vert \mathbf{x}-\mathbf{a}\Vert }=0\Rightarrow \mathbf{x}=\frac{1}{m}\left\{\sum _{i=1}^m{\mathbf{a}}_i+\sum _{i=1}^m{d}_i\frac{\mathbf{x}-{\mathbf{a}}_i}{\Vert \mathbf{x}-{\mathbf{a}}_i\Vert }\right\}$$
(ii)
$${\mathbf{x}}_{k+1}={\mathbf{x}}_k-\frac{1}{m}\nabla f\left({\mathbf{x}}_k\right)$$
是一个梯度方法，固定步长$\frac{1}{m}$

4.6. Another formulation of the source localization problem consists of minimizing the following objective function:
$$\text{ (SL2) }\underset{\mathbf{x}\in {\mathbb{R}}^n}{\min }\left\{f(\mathbf{x})\equiv \sum _{i=1}^m{\left({\Vert \mathbf{x}-{\mathbf{a}}_i\Vert }^2-d_i^2\right)}^2\right\}$$
This is of course a nonlinear least squares problem, and thus the Gauss-Newton method can be employed in order to solve it. We will assume that $n=2$.
(i)Show that as long as all the points ${\mathbf{a}}_1,\cdots ,{\mathbf{a}}_m$ do not reside on the same line in the plane, the method is well-defined, meaning that the linear least squares problem solved at each iteration has a unique solution.
(ii)Write a MATLAB function that implements the damped Gauss-Newton
method employed on problem (SL2) with a backtracking line search strategy with parameters $s=1,\alpha =\beta =0.5,ϵ=10^{-4}$.Run the function on the two-dimensional problem $\left(n=2\right)$ with 5 anchors $\left(m=5\right)$ and data generated by the MATLAB commands

randn('seed',317);
A=randn(2,5);
x=randn(2,1);
d=sqrt(sum((A-x*ones(1,5).^2)))+0.05*randn(1,5);
d=d';

The columns of the $2\times 5$ matrix $\mathbf{A}$ are the locations of the five sensors,$\mathbf{x}$ is the “true” location of the source, and $\mathbf{d}$ is the vector of noisy measurements between the source and the sensors. Compare your results (e.g., number of iterations) to the gradient method with backtracking and parameters $s=1,\alpha =\beta =0.5,ϵ=10^{-4}$.Start both methods with the initial vector ${\left(1000,-500\right)}^T$.
4.7. Let $f\left(\mathbf{x}\right)={\mathbf{x}}^T\mathbf{A}\mathbf{x}+2{\mathbf{b}}^T\mathbf{x}+c$,where $\mathbf{A}$ is a symmetric $n\times n$ matrix,$\mathbf{b}\in {\mathbb{R}}^n$,and $c\in \mathbb{R}$.Show that the smallest Lipschitz constant of $\nabla f$ is $2\Vert \mathbf{A}\Vert$.
解：
$$\nabla f\left(\mathbf{x}\right)=2\mathbf{A}\mathbf{x}+2\mathbf{b}$$
$$\Vert \nabla f\left(\mathbf{x}\right)-\nabla \left(\mathbf{y}\right)\Vert =\Vert 2\mathbf{A}\left(\mathbf{x}-\mathbf{y}\right)\Vert \le 2\Vert \mathbf{A}\Vert \Vert \mathbf{x}-\mathbf{y}\Vert$$
所以$L\le 2\Vert \mathbf{A}\Vert$

当$\mathbf{A}\mathbf{x}={\lambda }_1\mathbf{x}$时，$\Vert \nabla f\left(\mathbf{x}\right)-\nabla f\left(0\right)\Vert =2\Vert \mathbf{A}\mathbf{x}\Vert =2{\lambda }_1\Vert \mathbf{x}\Vert =2\Vert \mathbf{A}\Vert \Vert \mathbf{x}-0\Vert$
所以$L=2\Vert \mathbf{A}\Vert$

4.8. Let $f:{\mathbb{R}}^n\to \mathbb{R}$ be given by $f\left(\mathbf{x}\right)=\sqrt{1+\Vert \mathbf{x}{\Vert }^2}$.Show that $f\in C_1^{1,1}$.

解：
$$\nabla f\left(\mathbf{x}\right)=\frac{\mathbf{x}}{\sqrt{1+\Vert \mathbf{x}{\Vert }^2}}$$
$${\nabla }^{2}f\left(\mathbf{x}\right)=\frac{\left(1+{\mathbf{x}}^T\mathbf{x}\right)\mathbf{I}-\mathbf{x}{\mathbf{x}}^T}{{\left(1+\Vert \mathbf{x}{\Vert }^2\right)}^{\frac{3}{2}}}$$
注意到$\mathbf{x}{\mathbf{x}}^T$的特征值为$n-1$个0和1个${\mathbf{x}}^T\mathbf{x}$
于是$\Vert {\mathbf{x}}^T\mathbf{x}\mathbf{I}-\mathbf{x}{\mathbf{x}}^T\Vert ={\mathbf{x}}^T\mathbf{x}$
于是
$$\Vert {\nabla }^{2}f\left(\mathbf{x}\right)\Vert \le \frac{\Vert \mathbf{I}\Vert +\Vert {\mathbf{x}}^T\mathbf{x}\mathbf{I}-\mathbf{x}{\mathbf{x}}^T\Vert }{{\left(1+\Vert \mathbf{x}{\Vert }^2\right)}^{\frac{3}{2}}}=\frac{1+{\mathbf{x}}^T\mathbf{x}}{{\left(1+\Vert \mathbf{x}{\Vert }^2\right)}^{\frac{3}{2}}}\le 1$$

所以$L=1$

4.9. Let $f\in C_L^{1,1}\left({\mathbb{R}}^m\right)$, and let $\mathbf{A}\in {\mathbb{R}}^{m\times n},\mathbf{b}\in {\mathbb{R}}^m$.Show that the function $g:{\mathbb{R}}^n\to \mathbb{R}$ defined by $g\left(\mathbf{x}\right)=f\left(\mathbf{A}\mathbf{x}+\mathbf{b}\right)$ satisfies $g\in C_{\tilde{L}}^{1,1}\left({\mathbb{R}}^n\right)$,where $\tilde{L}=\Vert \mathbf{A}{\Vert }^{2}L$.
解：
$$\nabla g\left(\mathbf{x}\right)={\mathbf{A}}^T\nabla f\left(\mathbf{A}\mathbf{x}+b\right)$$
$$\Vert \nabla g\left(\mathbf{x}\right)-\nabla g\left(\mathbf{y}\right)\Vert =\Vert {\mathbf{A}}^T\left(\nabla f\left(\mathbf{A}\mathbf{x}+\mathbf{b}\right)-\nabla f\left(\mathbf{A}\mathbf{y}+\mathbf{b}\right)\right)\Vert \le \Vert {\mathbf{A}}^T\Vert \Vert \mathbf{A}\left(\mathbf{x}-\mathbf{y}\right)\Vert \le \Vert \mathbf{A}{\Vert }^{2}L\Vert \mathbf{x}-\mathbf{y}\Vert$$
所以$\tilde{L}=\Vert \mathbf{A}{\Vert }^{2}L$

4.10. Give an example of a function $f\in C_L^{1,1}\left(\mathbb{R}\right)$ and a starting point ${\mathbf{x}}_0\in \mathbb{R}$ such that the problem min $f\left(\mathbf{x}\right)$ has an optimal solution and the gradient method with constant stepsize $t=\frac{2}{L}$ diverges.
4.11. Suppose that $f\in C_L^{1,1}\left({\mathbb{R}}^n\right)$ and assume that ${\nabla }^{2}f\left(\mathbf{x}\right)⪰0$ for any $\mathbf{x}\in {\mathbb{R}}^n$. Suppose that the optimal value of the problem ${\min }_{\mathbf{x}\in {\mathbb{R}}^n}f\left(\mathbf{x}\right)$ is $f^{\ast }$. Let { x k } k ≥ 0 \left\{\mathbf{x}_k\right\}_{k\ge 0} {xk​}k≥0​ be thesequence generated by the gradient method with constant stepsize $\frac{1}{L}$.Show that if { x k } k ≥ 0 \left\{\mathbf{x}_k\right\}_{k\ge 0} {xk​}k≥0​ is bounded,then $f\left({\mathbf{x}}_k\right)\to f^{\ast }$ as $k\to \infty$.