Introduction to nonlinear optimization第二章习题

2.1. Find the global minimum and maximum points of the function $f(x,y)=x^2+y^2+2x-3y$ over the unit ball $S=B[0,1]=\left\{(x,y):x^2+y^2\le 1\right\}$.

解：

$f(x,y)=(x+1{)}^2+(y-\frac{3}{2}{)}^2-\frac{13}{4}$
显然最小值点和最大值点在$y=-\frac{3}{2}x$上
$$\{\begin{array}{l}y=-\frac{3}{2}x\\ x^2+y^2=1\end{array}\Rightarrow \{\begin{array}{l}x=\frac{2}{\sqrt{13}}\\ y=-\frac{3}{\sqrt{13}}\end{array}\text{或}\{\begin{array}{l}x=-\frac{2}{\sqrt{13}}\\ y=\frac{3}{\sqrt{13}}\end{array}$$
所以最小值点$(-\frac{2}{\sqrt{13}},\frac{3}{\sqrt{13}})$,最大值点$(\frac{2}{\sqrt{13}},-\frac{3}{\sqrt{13}})$

2.2. Let $\mathbf{a}\in {\mathbb{R}}^n$ be a nonzero vector. Show that the maximum of ${\mathbf{a}}^T\mathbf{x}$ over B [ 0 , 1 ] = { x ∈ R n : ∥ x ∥ ≤ 1 } B[0,1]=\left\{\mathbf{x}\in\mathbb{R}^n:\|\mathbf{x}\|\le 1\right\} B[0,1]={x∈Rn:∥x∥≤1} is attained at ${\mathbf{x}}^{\ast }=\frac{\mathbf{a}}{\Vert \mathbf{a}\Vert }$ and that the maximal value is $\Vert \mathbf{a}\Vert$.

解：
由柯西不等式
$${\mathbf{a}}^T\mathbf{x}\le \Vert \mathbf{a}\Vert \Vert \mathbf{x}\Vert \le \Vert \mathbf{a}\Vert$$
当且仅当$\mathbf{x}$与$\mathbf{a}$成比例时取等
并且$\Vert \mathbf{x}\Vert =1$,所以${\mathbf{x}}^{\ast }=\frac{\mathbf{a}}{\Vert \mathbf{a}\Vert }$

2.3. Find the global minimum and maximum points of the function $f(x,y)=2x-3y$ over the set $S=\left\{(x,y):2x^2+5y^2\le 1\right\}$.

解：
设$z=2x-3y$
于是$y=\frac{2}{3}x-\frac{z}{3}$

显然最小值点和最大值点在$y=-\frac{3}{2}x$上
$$\{\begin{array}{l}y=-\frac{3}{2}x\\ 2x^2+5y^2=1\end{array}\Rightarrow \{\begin{array}{l}x=\frac{2}{\sqrt{53}}\\ y=-\frac{3}{\sqrt{53}}\end{array}\text{或}\{\begin{array}{l}x=-\frac{2}{\sqrt{53}}\\ y=\frac{3}{\sqrt{53}}\end{array}$$

所以最小值点$(-\frac{2}{\sqrt{53}},\frac{3}{\sqrt{53}})$,最大值点$(\frac{2}{\sqrt{53}},-\frac{3}{\sqrt{53}})$

2.4. Show that if $\mathbf{A},\mathbf{B}$ are $n\times n$ positive semidefinite matrices, then their sum $\mathbf{A}+\mathbf{B}$ is also positive semidefinite.

解
$$\forall \mathbf{x}\ne 0,{\mathbf{x}}^T\mathbf{A}\mathbf{x}\ge 0,{\mathbf{x}}^T\mathbf{B}\mathbf{x}\ge 0$$
所以
$$\forall \mathbf{x}\ne 0,{\mathbf{x}}^T\left(\mathbf{A}+\mathbf{B}\right)\mathbf{x}\ge 0\Rightarrow \left(\mathbf{A}+\mathbf{B}\right)⪰0$$

2.5. Let $\mathbf{A}\in {\mathbb{R}}^{n\times n}$ and $\mathbf{B}\in {\mathbb{R}}^{n\times n}$ be two symmetric matrices. Prove that the following two claims are equivalent:
(i)$\mathbf{A}$ and $\mathbf{B}$ are positive semidefinite.
(ii)$\left(\begin{array}{cc}\mathbf{A}& 0_{n\times m}\\ 0_{m\times n}& \mathbf{B}\end{array}\right)$ is positive semidefinite.

解：
$(i)\Rightarrow (ii)$
因为
$$\begin{gathered} \forall \mathbf{x}\ne 0,{\mathbf{x}}^T\mathbf{A}\mathbf{x}\ge 0 \\ \forall \mathbf{y}\ne 0,{\mathbf{y}}^T\mathbf{B}\mathbf{y}\ge 0 \end{gathered}$$
于是
$$\forall \mathbf{z}=\left(\begin{array}{c}\mathbf{x}\\ \mathbf{y}\end{array}\right)\ne 0,{\mathbf{z}}^T\left(\begin{array}{cc}\mathbf{A}& 0_{n\times m}\\ 0_{m\times n}& \mathbf{B}\end{array}\right)\mathbf{z}={\mathbf{x}}^T\mathbf{A}\mathbf{x}+{\mathbf{y}}^T\mathbf{B}\mathbf{y}\ge 0$$
所以
$$\left(\begin{array}{cc}\mathbf{A}& 0_{n\times m}\\ 0_{m\times n}& \mathbf{B}\end{array}\right)⪰0$$

$(ii)\Rightarrow (i)$
$$\forall \mathbf{z}=\left(\begin{array}{c}\mathbf{x}\\ 0\end{array}\right)\ne 0,{\mathbf{z}}^T\left(\begin{array}{cc}\mathbf{A}& 0_{n\times m}\\ 0_{m\times n}& \mathbf{B}\end{array}\right)\mathbf{z}={\mathbf{x}}^T\mathbf{A}\mathbf{x}\ge 0\Rightarrow \mathbf{A}⪰0$$

$$\forall \mathbf{z}=\left(\begin{array}{c}0\\ \mathbf{y}\end{array}\right)\ne 0,{\mathbf{z}}^T\left(\begin{array}{cc}\mathbf{A}& 0_{n\times m}\\ 0_{m\times n}& \mathbf{B}\end{array}\right)\mathbf{z}={\mathbf{y}}^T\mathbf{B}\mathbf{y}\ge 0\Rightarrow \mathbf{B}⪰0$$

2.6. Let $\mathbf{B}\in {\mathbb{R}}^{n\times k}$ and let $\mathbf{A}=\mathbf{B}{\mathbf{B}}^T$.
(i)Prove $\mathbf{A}$ is positive semidefinite.
(ii)Prove that $\mathbf{A}$ is positive definite if and only if $\mathbf{B}$ has a full row rank.

解：
(i)
$\forall \mathbf{x}\ne 0,{\mathbf{x}}^T\mathbf{A}\mathbf{x}={\mathbf{x}}^T\mathbf{B}{\mathbf{B}}^T\mathbf{x}=\Vert {\mathbf{B}}^T\mathbf{x}{\Vert }^2\ge 0\Rightarrow \mathbf{A}⪰0$

(ii)
利用$r(\mathbf{A})=r({\mathbf{A}}^T)=r({\mathbf{A}}^T\mathbf{A})$
显然成立

2.7.
(i) Let $\mathbf{A}$ be an $n\times n$ symmetric matrix. Show that $\mathbf{A}$ is positive semidefinite if and only if there exists a matrix $\mathbf{B}\in {\mathbb{R}}^{n\times n}$ such that $\mathbf{A}=\mathbf{B}{\mathbf{B}}^T$.
(ii) Let $\mathbf{x}\in {\mathbb{R}}^n$ and Let $\mathbf{A}$ be defined as
$${\mathbf{A}}_{ij}={\mathbf{x}}_i{\mathbf{x}}_j,i,j=1,2,\cdots ,n.$$
Show that $\mathbf{A}$ is positive semidefinite and that it is not a positive definite matrix when $n>1$.

解：
(i)实对称矩阵必可对角化
存在正交矩阵$\mathbf{P}$,使得$\mathbf{A}=\mathbf{P}\Lambda {\mathbf{P}}^T$
其中$\Lambda$是对角线为$\mathbf{A}$的特征值的对角矩阵

如果$\mathbf{A}⪰0$
$\mathbf{A}=\mathbf{P}\Lambda {\mathbf{P}}^T=\mathbf{P}{\Lambda }^{\frac{1}{2}}{\Lambda }^{\frac{1}{2}}{\mathbf{P}}^T$
取$\mathbf{B}={\Lambda }^{\frac{1}{2}}{\mathbf{P}}^T$,即可

如果$\mathbf{A}=\mathbf{B}{\mathbf{B}}^T$,由上一题，$\mathbf{A}⪰0$

(ii)
$\mathbf{A}=\mathbf{x}{\mathbf{x}}^T$
$$\forall \mathbf{y}\ne 0,{\mathbf{y}}^T\mathbf{A}\mathbf{y}=\Vert {\mathbf{x}}^T\mathbf{y}{\Vert }^2\ge 0\Rightarrow \mathbf{A}⪰0$$

取$\mathbf{x}={\left(1,0\right)}^T$
$\mathbf{A}=\left(\begin{array}{cc}1& 0\\ 0& 0\end{array}\right)⪰0$,并不是正定

2.8. Let $\mathbf{Q}\in {\mathbb{R}}^{n\times n}$ be a positive definite matrix. Show that the “Q-norm” defined by
$$\Vert \mathbf{x}{\Vert }_{\mathbf{Q}}=\sqrt{{\mathbf{x}}^T\mathbf{Q}\mathbf{x}}$$
is indeed a norm.

解：
$\forall \mathbf{x}\ne 0,{\mathbf{x}}^T\mathbf{Q}\mathbf{x}>0$

所以$\forall \mathbf{x}\ne 0,\Vert \mathbf{x}{\Vert }_{\mathbf{Q}}>0$,满足非负性

$\forall k\in \mathbb{R},\Vert k\mathbf{x}{\Vert }_{\mathbf{Q}}=\left|k\right|\Vert \mathbf{x}{\Vert }_{\mathbf{Q}}$,满足齐次性

$\forall x,y\in {\mathbb{R}}_{++},\sqrt{x+y}<\sqrt{x}+\sqrt{y}$
所以
$$\Vert \mathbf{x}+\mathbf{y}{\Vert }_{\mathbf{Q}}=\sqrt{{\mathbf{x}}^T\mathbf{Q}\mathbf{x}+{\mathbf{y}}^T\mathbf{Q}\mathbf{y}}\le \sqrt{{\mathbf{x}}^T\mathbf{Q}\mathbf{x}}+\sqrt{{\mathbf{y}}^T\mathbf{Q}\mathbf{y}}=\Vert \mathbf{x}{\Vert }_{\mathbf{Q}}+\Vert \mathbf{y}{\Vert }_{\mathbf{Q}}$$

2.9. Let $\mathbf{A}$ be an $n\times n$ positive semidefinite matrix.
(i)Show that for any $i\ne j$
$${\mathbf{A}}_{ii}{\mathbf{A}}_{jj}\ge {\mathbf{A}}_{ij}^2$$
(ii)Show that if for some i ∈ { 1 , 2 , ⋯   , n } A i i = 0 i\in\left\{1,2,\cdots,n\right\}\mathbf{A}_{ii}=0 i∈{1,2,⋯,n}Aii​=0,then the ith row of $\mathbf{A}$ consists of zeros.

解：
(i)
设$\mathbf{x}=x_1{\mathbf{e}}_i+x_2{\mathbf{e}}_j$
$$\begin{array}{rl}{\mathbf{x}}^T\mathbf{A}\mathbf{x}& ={\mathbf{A}}_{ii}{x}_1^2+2{\mathbf{A}}_{ij}^2{x}_1{x}_2+{\mathbf{A}}_{jj}{x}_2^2\\ & ={\left(\begin{array}{c}{x}_1\\ x_2\end{array}\right)}^T\left(\begin{array}{cc}{\mathbf{A}}_{ii}& {\mathbf{A}}_{ij}\\ {\mathbf{A}}_{ij}& {\mathbf{A}}_{jj}\end{array}\right)\left(\begin{array}{c}{x}_1\\ x_2\end{array}\right)\\ & \ge 0\end{array}$$
所以
$$\left(\begin{array}{cc}{\mathbf{A}}_{ii}& {\mathbf{A}}_{ij}\\ {\mathbf{A}}_{ij}& {\mathbf{A}}_{jj}\end{array}\right)⪰0\Rightarrow {\mathbf{A}}_{ii}{\mathbf{A}}_{jj}\ge {\mathbf{A}}_{ij}^2$$
(ii)
$${\mathbf{A}}_{ij}^2\le {\mathbf{A}}_{ii}{\mathbf{A}}_{jj}=0\Rightarrow {\mathbf{A}}_{ij}=0$$
所以第i行为0

2.10. Let ${\mathbf{A}}^{\alpha }$ be the $n\times n$ matrix $\left(n>1\right)$ defined by
$${\mathbf{A}}_{ij}^{\alpha }=\{\begin{array}{ll}\alpha ,& i=j,\\ 1,& i\ne j.\end{array}$$
Show that ${\mathbf{A}}^{\alpha }$ is positive semidefinite if and only if $\alpha \ge 1$

解:
$$\begin{array}{rl}& \left|\lambda \mathbf{I}-{\mathbf{A}}^{\alpha }\right|\\ & =\left|\begin{array}{ccccc}\lambda -\alpha & -1& -1& \cdots & -1\\ -1& \lambda -\alpha & -1& \cdots & -1\\ -1& -1& \lambda -\alpha & \cdots & -1\\ ⋮& ⋮& & \ddots & \\ -1& -1& -1& \cdots & \lambda -\alpha \end{array}\right|\\ & =\left|\begin{array}{ccccc}\lambda -\alpha -n+1& -1& -1& \cdots & -1\\ \lambda -\alpha -n+1& \lambda -\alpha & -1& \cdots & -1\\ \lambda -\alpha -n+1& -1& \lambda -\alpha & \cdots & -1\\ ⋮& ⋮& & \ddots & \\ \lambda -\alpha -n+1& -1& -1& \cdots & \lambda -\alpha \end{array}\right|\\ & =(\lambda -\alpha -n+1)\left|\begin{array}{ccccc}1& -1& -1& \cdots & -1\\ 1& \lambda -\alpha & -1& \cdots & -1\\ 1& -1& \lambda -\alpha & \cdots & -1\\ ⋮& ⋮& & \ddots & \\ 1& -1& -1& \cdots & \lambda -\alpha \end{array}\right|\\ & =\left(\lambda -\alpha -n+1\right)\left|\begin{array}{ccccc}1& 0& 0& \cdots & 0\\ 0& \lambda -\alpha +1& 0& \cdots & 0\\ 0& 0& \lambda -\alpha +1& \cdots & 0\\ ⋮& ⋮& & \ddots & \\ 0& 0& 0& \cdots & \lambda -\alpha +1\end{array}\right|\\ & =\left(\lambda -\alpha -n+1\right){\left(\lambda -\alpha +1\right)}^{n-1}\\ & =0\end{array}$$
所以特征值为$\alpha +n-1$和n-1个$\alpha -1$
$$\{\begin{array}{l}\alpha +n-1\ge 0\\ \alpha -1\ge 0\end{array}\iff \alpha \ge 1$$
所以${\mathbf{A}}^{\alpha }⪰0\iff \alpha \ge 1$

2.11. Let $\mathbf{d}\in {\Delta }_n$ (${\Delta }_n$ being the unit-simplex).Show that the $n\times n$ matrix $\mathbf{A}$ defined by
$${\mathbf{A}}_{ij}=\{\begin{array}{ll}{d}_i-d_i^2,& i=j,\\ -d_i{d}_j,& i\ne j,\end{array}$$
is positive semidefinite.

解：
$$\begin{array}{rl}& \left|A_{ii}\right|-\sum _{i\ne j}\left|{\mathbf{A}}_{ij}\right|\\ & =d_i-d_i^2-\sum _{i\ne j}{d}_i{d}_j\\ & =d_i-\sum _{j=1}^n{d}_i{d}_j\\ & =d_i-d_i\sum _{j=1}^n{d}_j\\ & =d_i-d_i\\ & =0\end{array}$$
所以
$$\left|A_{ii}\right|\ge \sum _{i\ne j}\left|{\mathbf{A}}_{ij}\right|$$

所以$\mathbf{A}$是对角占优矩阵
又${\mathbf{A}}_{ii}\ge 0$
所以$\mathbf{A}⪰0$

2.12. Prove that a $2\times 2$ matrix $\mathbf{A}$ is negative semidefinite if and only if $Tr(\mathbf{A})\le 0$ and $det(\mathbf{A})\ge 0$.

解：
$$\{\begin{array}{l}Tr(\mathbf{A})={\lambda }_1+{\lambda }_2\le 0\\ det(\mathbf{A})={\lambda }_1{\lambda }_2\ge 0\end{array}\iff {\lambda }_1,{\lambda }_2\le 0\iff \mathbf{A}⪯0$$

2.13.For each of the following matrices determine whether they are positive/negative semidefinite/ definite or indefinite:
(i)$\left(\begin{array}{cccc}2& 2& 0& 0\\ 2& 2& 0& 0\\ 0& 0& 3& 1\\ 0& 0& 1& 3\end{array}\right)$
(ii)$\left(\begin{array}{ccc}2& 2& 2\\ 2& 3& 3\\ 2& 3& 3\end{array}\right)$
(iii)$\left(\begin{array}{ccc}2& 1& 3\\ 1& 2& 1\\ 3& 1& 2\end{array}\right)$
(iv)$\left(\begin{array}{ccc}-5& 1& 1\\ 1& -7& 1\\ 1& 1& -5\end{array}\right)$

解：
(i)$\mathbf{A}$是对角占优矩阵，对角线元素非负，所以$\mathbf{A}⪰0$
(ii)
$$\begin{array}{rl}& \left|\lambda \mathbf{I}-\mathbf{B}\right|\\ & =\left|\begin{array}{ccc}\lambda -2& -2& -2\\ -2& \lambda -3& -3\\ -2& -3& \lambda -3\end{array}\right|\\ & =\left|\begin{array}{ccc}\lambda -2& -2& -2\\ -2& \lambda -3& -3\\ 0& -\lambda & \lambda \end{array}\right|\\ & =\left|\begin{array}{ccc}\lambda -2& -4& -2\\ -2& \lambda -6& -3\\ 0& 0& \lambda \end{array}\right|\\ & =\lambda ({\lambda }^2-8\lambda +12-8)\\ & =\lambda ({\lambda }^2-8\lambda +4)\\ & =\lambda \left(\lambda -(4+2\sqrt{3})\right)\left(\lambda -(4-2\sqrt{3})\right)\end{array}$$
所以$\mathbf{B}⪰0$
(iii)
$$\begin{array}{rl}& \left|\lambda \mathbf{I}-\mathbf{B}\right|\\ & =\left|\begin{array}{ccc}\lambda -2& -1& -3\\ -1& \lambda -2& -1\\ -3& -1& \lambda -2\end{array}\right|\\ & =\left|\begin{array}{ccc}\lambda +1& -1& -3\\ 0& \lambda -2& -1\\ -\lambda -1& -1& \lambda -2\end{array}\right|\\ & =\left|\begin{array}{ccc}\lambda +1& -1& -3\\ 0& \lambda -2& -1\\ 0& -2& \lambda -5\end{array}\right|\\ & =\left(\lambda +1\right)\left({\lambda }^2-7\lambda +8\right)\\ & =\left(\lambda +1\right)\left(\lambda -\frac{7+\sqrt{17}}{2}\right)\left(\lambda -\frac{7-\sqrt{17}}{2}\right)\end{array}$$
特征值有正有负，所以$\mathbf{C}$不定

(iv)
因为$-\mathbf{D}$是严格对角占优矩阵，且对角线元素是正的，所以
$$-\mathbf{D}\succ 0\Rightarrow \mathbf{D}\prec 0$$

2.14. (Schur complement lemma) Let
$$\mathbf{D}=\left(\begin{array}{cc}\mathbf{A}& \mathbf{b}\\ {\mathbf{b}}^T& c\end{array}\right)$$
where $\mathbf{A}\in {\mathbb{R}}^{n\times n},\mathbf{b}\in {\mathbb{R}}^n,c\in \mathbb{R}$.Suppose that $\mathbf{A}\succ 0$.Prove that $\mathbf{D}⪰0$ if and only if $c-{\mathbf{b}}^T{\mathbf{A}}^{-1}\mathbf{b}\ge 0$.

解：
设
$$\mathbf{T}=\left(\begin{array}{cc}\mathbf{A}& 0\\ 0& c-{\mathbf{b}}^T{\mathbf{A}}^{-1}\mathbf{b}\end{array}\right)$$
$$\mathbf{N}=\left(\begin{array}{cc}\mathbf{I}& 0\\ {\mathbf{b}}^T{\mathbf{A}}^{-1}& 1\end{array}\right)$$
$$\mathbf{D}=\mathbf{N}\mathbf{T}{\mathbf{N}}^T$$
于是显然成立

2.15. For each of the following functions, determine whether it is coercive or not:
(i)$f\left(x_1,x_2\right)=x_1^4+x_2^4$
(ii)$f\left(x_1,x_2\right)=e^{x_1^2}+e^{x_2^2}-x_1^{200}-x_2^{200}$
(iii)$f\left(x_1,x_2\right)=2x_1^2-8x_1{x}_2+x_2^2$
(iv)$f\left(x_1,x_2\right)=4x_1^2+2x_1{x}_2+2x_2^2$
(v)$f\left(x_1,x_2,x_3\right)=x_1^3+x_2^3+x_3^3$
(vi)$f\left(x_1,x_2\right)=x_1^2-2x_1{x}_2^2+x_2^4$
(vii)$f\left(\mathbf{x}\right)=\frac{{\mathbf{x}}^T\mathbf{A}\mathbf{x}}{\Vert \mathbf{x}\Vert +1}$,where $\mathbf{A}\in {\mathbb{R}}^{n\times n}$ is positive definite.

解：
(i)
当$\Vert \mathbf{x}\Vert \to \infty$,
$f\left(x_1,x_2\right)\ge \Vert \mathbf{x}{\Vert }^2\to \infty$
所以是

(ii)
$e^{x_1^2}+e^{x_2^2}$占据主导，所以是

(iii)
$$\mathbf{A}=\left(\begin{array}{cc}2& -4\\ -4& 1\end{array}\right)$$
并不正定，所以不是

(iv)
$$\mathbf{A}=\left(\begin{array}{cc}4& 1\\ 1& 2\end{array}\right)\succ 0$$
所以是

(v)
当$x_1\to -\infty ,x_2\to -\infty ,x_3\to -\infty$时
$f\left(x_1,x_2,x_3\right)\to -\infty$
所以不是

(vi)
$f\left(x_1,x_2\right)={\left(x_1-x_2^2\right)}^2$
取$\mathbf{v}={\left(t,\sqrt{t}\right)}^T$
当$t\to \infty$时，$\Vert \mathbf{v}\Vert \to \infty$
但是$f\left(\mathbf{v}\right)\to 0$
所以不是

(vii)
$f\left(\mathbf{x}\right)\ge \frac{{\lambda }_{min}\Vert \mathbf{x}{\Vert }^2}{\Vert \mathbf{x}\Vert +1}$

当$\Vert \mathbf{x}\Vert \to \infty$时，
$\frac{{\lambda }_1\Vert \mathbf{x}\Vert }{\Vert \mathbf{x}\Vert +1}\to \infty$
所以是

2.15. Find a function $f:{\mathbb{R}}^2\to \mathbb{R}$ which is not coercive and satisfies that for any $\alpha \in \mathbb{R}$
$$\underset{|x_1|\to \infty }{\lim }f\left(x_1,\alpha x_1\right)=\underset{|x_2|\to \infty }{\lim }f\left(\alpha x_2,x_2\right)=\infty$$

解：
$f\left(x_1,x_2\right)={\left(x_1-x_2^2\right)}^2$

2.17. For each of the following functions, find all the stationary points and classify them according to whether they are saddle points, strict/nonstrict local/global minimum/ maximum points:
(i)$f\left(x_1,x_2\right)={\left(4x_1^2-x_2\right)}^2$
(ii)$f\left(x_1,x_2,x_3\right)=x_1^4-2x_1^2+x_2^2+2x_2{x}_3+2x_3^2$
(iii)$f\left(x_1,x_2\right)=2x_2^3-6x_2^2+3x_1^2{x}_2$
(iv)$f\left(x_1,x_2\right)=x_1^4+2x_1^2{x}_2+x_2^2-4x_1^2-8x_1-8x_2$
(v)$f\left(x_1,x_2\right)={\left(x_1-2x_2\right)}^4+64x_1{x}_2$
(vi)$f\left(x_1,x_2\right)=2x_1^2+3x_2^2-2x_1{x}_2+2x_1-3x_2$
(vii)$f\left(x_1,x_2\right)=x_1^2+4x_1{x}_2+x_2^2+x_1-x_2$

解：
(i)
$$\nabla f=\left(\begin{array}{c}16x_1\left(4x_1^2-x_2\right)\\ -2\left(4x_1^2-x_2\right)\end{array}\right)=0\Rightarrow 4x_1^2=x_2$$
$$f\left(x_1,x_2\right)\ge 0=f\left(x_1,4x_1^2\right)$$
所以$\left(x_1,4x_1^2\right)$上的点是全局最小值点

或者
$${\nabla }^{2}f\left(x_1,4x_1^2\right)=\left(\begin{array}{cc}16\left(12x_1^2-x_2\right)& -16x_1\\ -16x_1& 2x_2\end{array}\right)=\left(\begin{array}{cc}128x_1^2& -16x_1\\ -16x_1& 2x_2\end{array}\right)⪰0$$
因此也是全局最小值点

(ii)
$$\nabla f=\left(\begin{array}{c}4x_1^3-4x_1\\ 2x_2+2x_3\\ 2x_2+4x_3\end{array}\right)=0\Rightarrow \left(\begin{array}{c}{x}_1\\ x_2\\ x_3\end{array}\right)=\left(\begin{array}{c}0\\ 0\\ 0\end{array}\right)or\left(\begin{array}{c}1\\ 0\\ 0\end{array}\right)or\left(\begin{array}{c}-1\\ 0\\ 0\end{array}\right)$$
$${\nabla }^{2}f=\left(\begin{array}{ccc}12x_1^2-4& 0& 0\\ 0& 2& 2\\ 0& 2& 4\end{array}\right)$$
${\nabla }^{2}f\left(0,0,0\right)$不定,所以$\left(0,0,0\right)$是鞍点
${\nabla }^{2}f\left(1,0,0\right)\succ 0$,所以$\left(1,0,0\right)$是严格局部最小值点
${\nabla }^{2}f\left(-1,0,0\right)\succ 0$,所以$\left(-1,0,0\right)$是严格局部最小值点
(iii)
$$\nabla f=\left(\begin{array}{c}6x_1{x}_2\\ 6x_2^2-12x_2+3x_1^2\end{array}\right)=0\Rightarrow \left(\begin{array}{c}{x}_1\\ x_2\end{array}\right)=\left(\begin{array}{c}0\\ 0\end{array}\right)or\left(\begin{array}{c}0\\ 2\end{array}\right)$$
$${\nabla }^{2}f=6\left(\begin{array}{cc}{x}_2& x_1\\ x_1& 2\left(x_2-1\right)\end{array}\right)$$
${\nabla }^{2}f\left(0,0\right)$不定，所以$\left(0,0\right)$是鞍点
${\nabla }^{2}f\left(0,2\right)\succ 0$，所以$\left(0,2\right)$是严格局部最小值点
(iv)
$$\nabla f=\left(\begin{array}{c}4x_1^3+4x_1{x}_2-8x_1-8\\ 2x_1^2+2x_2-8\end{array}\right)=0\Rightarrow \left(\begin{array}{c}{x}_1\\ x_2\end{array}\right)=\left(\begin{array}{c}1\\ 3\end{array}\right)$$
$${\nabla }^{2}f\left(1,3\right)=2\left(\begin{array}{cc}6x_1^2+2x_2-4& 2x_1\\ 2x_1& 1\end{array}\right)=2\left(\begin{array}{cc}8& 2\\ 2& 1\end{array}\right)\succ 0$$
所以$\left(1,3\right)$是严格局部最小值点
又因为$f\left(x_1,x_2\right)={\left(x_1^2+x_2-4\right)}^2+{\left(x_1-1\right)}^2-20$
所以是严格全局最小值点
(v)
$$\nabla f=\left(\begin{array}{c}4{\left(x_1-2x_2\right)}^3+64x_2\\ -8{\left(x_1-2x_2\right)}^3+64x_1\end{array}\right)=0\Rightarrow \left(\begin{array}{c}{x}_1\\ x_2\end{array}\right)=\left(\begin{array}{c}0\\ 0\end{array}\right)or\left(\begin{array}{c}-1\\ \frac{1}{2}\end{array}\right)or\left(\begin{array}{c}1\\ -\frac{1}{2}\end{array}\right)$$
$${\nabla }^{2}f=4\left(\begin{array}{cc}3{\left(x_1-2x_2\right)}^2& -6{\left(x_1-2x_2\right)}^2+16\\ -6{\left(x_1-2x_2\right)}^2+16& 12{\left(x_1-2x_2\right)}^2\end{array}\right)$$
${\nabla }^{2}f\left(0,0\right)$不定,所以$\left(0,0\right)$是鞍点
${\nabla }^{2}f\left(-1,\frac{1}{2}\right)\succ 0$,所以$\left(-1,\frac{1}{2}\right)$是严格局部最小值点
${\nabla }^{2}f\left(1,-\frac{1}{2}\right)\succ 0$,所以$\left(1,-\frac{1}{2}\right)$是严格局部最小值点
(vi)
$$\nabla f=\left(\begin{array}{c}4x_1-2x_2+2\\ 6x_2-2x_1-3\end{array}\right)=0\Rightarrow \left(\begin{array}{c}{x}_1\\ x_2\end{array}\right)=\left(\begin{array}{c}-\frac{3}{10}\\ \frac{2}{5}\end{array}\right)$$
$${\nabla }^{2}f=\left(\begin{array}{cc}4& -2\\ -2& 6\end{array}\right)\succ 0$$
所以$\left(-\frac{3}{10},\frac{2}{5}\right)$是严格全局最小值点
(vii)
$$\nabla f=\left(\begin{array}{c}2x_1+4x_2+1\\ 4x_1+2x_2-1\end{array}\right)=0\Rightarrow \left(\begin{array}{c}{x}_1\\ x_2\end{array}\right)=\left(\begin{array}{c}\frac{1}{2}\\ -\frac{1}{2}\end{array}\right)$$
$${\nabla }^{2}f=\left(\begin{array}{cc}2& 4\\ 4& 2\end{array}\right)$$
${\nabla }^{2}f$不定，所以$\left(\frac{1}{2},-\frac{1}{2}\right)$是鞍点
2.18. Let $f$ be twice continuously differentiable function over ${\mathbb{R}}^n$. Suppose that ${\nabla }^{2}f\left(\mathbf{x}\right)\succ 0$ for any $\mathbf{x}\in {\mathbb{R}}^n$.Prove that a stationary point of $f$ is necessarily a strict global minimum point.

解：
（应该是说如果是驻点，则是严格全局最小点吧）

设${\mathbf{x}}^{\ast }$是一个驻点
$$f\left(\mathbf{x}\right)-f\left({\mathbf{x}}^{\ast }\right)=\frac{1}{2}{\left(\mathbf{x}-{\mathbf{x}}^{\ast }\right)}^T{\nabla }^{2}f\left(\mathbf{z}\right)\left(\mathbf{x}-{\mathbf{x}}^{\ast }\right)>0$$
其中$\mathbf{x}\ne {\mathbf{x}}^{\ast }$,$\mathbf{z}$介于$\mathbf{x},{\mathbf{x}}^{\ast }$之间

可以得到，这个驻点严格全局最小值
且是唯一的，否则与海瑟矩阵正定矛盾

2.19. Let $f\left(\mathbf{x}\right)={\mathbf{x}}^T\mathbf{A}\mathbf{x}+2{\mathbf{b}}^T\mathbf{x}+c$, where $\mathbf{A}\in {\mathbb{R}}^{n\times n}$ is symmetric,$\mathbf{b}\in {\mathbf{R}}^n$, and $c\in \mathbb{R}$. Suppose that $\mathbf{A}⪰0$.Show that f is bounded below over ${\mathbb{R}}^n$ if and only if b ∈ R a n g e ( A ) = { A y : y ∈ R n } \mathbf{b}\in Range\left(\mathbf{A}\right)=\left\{\mathbf{Ay}:\mathbf{y}\in\mathbb{R}^n\right\} b∈Range(A)={Ay:y∈Rn}.

(A function f is bounded below over a set $C$ if there exists a constant $\alpha$ such that $f\left(\mathbf{x}\right)\ge \alpha$ for any $\mathbf{x}\in C$)

解：

$$\begin{gathered} f\left(\mathbf{x}\right)={\mathbf{x}}^T\mathbf{A}\mathbf{x}+2{\mathbf{b}}^T\mathbf{x}+c \\ \nabla f\left(\mathbf{x}\right)=2\mathbf{A}\mathbf{x}+2\mathbf{b} \\ \nabla f^2\left(\mathbf{x}\right)=2\mathbf{A} \end{gathered}$$

如果 b ∈ R a n g e ( A ) = { A y : y ∈ R n } \mathbf{b}\in Range\left(\mathbf{A}\right)=\left\{\mathbf{Ay}:\mathbf{y}\in\mathbb{R}^n\right\} b∈Range(A)={Ay:y∈Rn},
说明$\nabla f\left(\mathbf{x}\right)=0$有解，则$f$存在全局最小值，所以有下界

如果$f$有下界，假设$\mathbf{b}\notin Range\left(\mathbf{A}\right)$
则$\mathbf{b}/\perp N\left({\mathbf{A}}^T\right)=N\left(\mathbf{A}\right)$(其实我也不确定这个对不对)
于是存在$\mathbf{y}$,使得${\mathbf{y}}^T\mathbf{A}\mathbf{y}=0,{\mathbf{b}}^T\mathbf{y}<0$
当$\lambda \to +\infty$时，有$f\left(\lambda \mathbf{y}\right)\to -\infty$,矛盾
所以 b ∈ R a n g e ( A ) = { A y : y ∈ R n } \mathbf{b}\in Range\left(\mathbf{A}\right)=\left\{\mathbf{Ay}:\mathbf{y}\in\mathbb{R}^n\right\} b∈Range(A)={Ay:y∈Rn}