Introduction to nonlinear optimization第一章习题

1.1 Show that $\Vert \cdot {\Vert }_{\frac{1}{2}}$ is not a norm.

解：
$\mathbf{x}=\left(\begin{array}{c}1\\ 0\end{array}\right),\mathbf{y}=\left(\begin{array}{c}1\\ 1\end{array}\right)$
$$\Vert \mathbf{x}+\mathbf{y}{\Vert }_{\frac{1}{2}}=(\sqrt{2}+1{)}^2=3+2\sqrt{2}$$
$$\Vert \mathbf{x}{\Vert }_{\frac{1}{2}}+\Vert \mathbf{y}{\Vert }_{\frac{1}{2}}=1+\sqrt{2}<\Vert \mathbf{x}+\mathbf{y}{\Vert }_{\frac{1}{2}}$$
所以不是

1.2 Prove that for any $\mathbf{x}\in {\mathbb{R}}^n$ one has
$$\Vert \mathbf{x}{\Vert }_{\infty }=\underset{p\to \infty }{\lim }\Vert \mathbf{x}{\Vert }_p$$

解：
$$\begin{array}{rl}& \underset{p\to \infty }{\lim }\Vert \mathbf{x}{\Vert }_p\\ & =\underset{p\to \infty }{\lim }\sqrt[p]{\sum _{i=1}^n{\left|x_i\right|}^p}\\ & =\max \left|x_i\right|\\ & =\Vert \mathbf{x}{\Vert }_{\infty }\end{array}$$
1.3 Show that for any $\mathbf{x},\mathbf{y},\mathbf{z}\in {\mathbb{R}}^n$
$$\Vert \mathbf{x}-\mathbf{z}\Vert \le \Vert \mathbf{x}-\mathbf{y}\Vert +\Vert \mathbf{y}-\mathbf{z}\Vert$$

解：
$$\begin{array}{rl}& \Vert \mathbf{x}-\mathbf{z}\Vert \\ & =\Vert \mathbf{x}-\mathbf{y}+\mathbf{y}-\mathbf{z}\Vert \\ & \le \Vert \mathbf{x}-\mathbf{y}\Vert +\Vert \mathbf{y}-\mathbf{z}\Vert \end{array}$$
1.4 Prove the Cauchy–Schwarz inequality (Lemma 1.5).Show that equality holds if and only if the vectors $\mathbf{x}$ and $\mathbf{y}$ are linearly dependent.

解.
$$\begin{array}{rl}& \sum _{i=1}^n(x_{i}z-y_i{)}^2\\ & =\sum _{i=1}^n(x_i^2{z}^2-2x_i{y}_{i}z+y_i^2)\\ & =\left(\sum _{i=1}^n{x}_i^2\right)z^2-2\left(\sum _{i=1}^n{x}_i{y}_i\right)z+\sum _{i=1}^n{y}_i^2\\ & \ge 0\end{array}$$
$$\begin{gathered} \Rightarrow \Delta =4{\left(\sum _{i=1}^n{x}_i{y}_i\right)}^2-4\left(\sum _{i=1}^n{x}_i^2\right)\left(\sum _{i=1}^n{y}_i^2\right)\le 0 \\ \Rightarrow \left|{\mathbf{x}}^T\mathbf{y}\right|\le \Vert \mathbf{x}\Vert \Vert \mathbf{y}\Vert \end{gathered}$$
当且仅当$z\mathbf{x}=\mathbf{y}$时，$\Delta =0$,$\left|{\mathbf{x}}^T\mathbf{y}\right|=\Vert \mathbf{x}\Vert \Vert \mathbf{y}\Vert$

1.5 Suppose that ${\mathbb{R}}^m$ and ${\mathbb{R}}^n$ are equipped with norms $\Vert \cdot {\Vert }_b$ and $\Vert \cdot {\Vert }_a$, respectively.Show that the induced matrix norm $\Vert \cdot {\Vert }_{a,b}$satisfies the triangle inequality. That is,for any $\mathbf{A},\mathbf{B}\in {\mathbb{R}}^{m\times n}$ the inequality
$$\Vert \mathbf{A}+\mathbf{B}{\Vert }_{a,b}\le \Vert \mathbf{A}{\Vert }_{a,b}+\Vert \mathbf{B}{\Vert }_{a,b}$$
holds.

解：
$$\begin{array}{rl}& \Vert \mathbf{A}+\mathbf{B}{\Vert }_{a,b}\\ & =\underset{\Vert \mathbf{x}{\Vert }_a\le 1}{\max }\frac{\Vert \left(\mathbf{A}+\mathbf{B}\right)\mathbf{x}{\Vert }_b}{\Vert \mathbf{x}{\Vert }_a}\\ & \le \underset{\Vert \mathbf{x}{\Vert }_a\le 1}{\max }\frac{\Vert \mathbf{A}\mathbf{x}{\Vert }_b+\Vert \mathbf{B}\mathbf{x}{\Vert }_b}{\Vert \mathbf{x}{\Vert }_a}\\ & \le \underset{\Vert \mathbf{x}{\Vert }_a\le 1}{\max }\frac{\Vert \mathbf{A}\mathbf{x}{\Vert }_b}{\Vert \mathbf{x}{\Vert }_a}+\underset{\Vert \mathbf{y}{\Vert }_a\le 1}{\max }\frac{\Vert \mathbf{B}\mathbf{y}{\Vert }_b}{\Vert \mathbf{y}{\Vert }_a}\\ & =\Vert \mathbf{A}{\Vert }_{a,b}+\Vert \mathbf{B}{\Vert }_{a,b}\end{array}$$

1.6 Let $\Vert \cdot \Vert$ be a norm on ${\mathbb{R}}^n$.Show that the norm function $f\left(\mathbf{x}\right)=\Vert \mathbf{x}\Vert$ is a continuous function over ${\mathbb{R}}^n$.

解：
对于$\forall ϵ>0,\exists \delta >0$,
当$0<\Vert \mathbf{x}-\mathbf{y}\Vert <\delta$时
$\left|f\left(\mathbf{x}\right)-f\left(\mathbf{y}\right)\right|=\left|\Vert \mathbf{x}\Vert -\Vert \mathbf{y}\Vert \right|\le \Vert \mathbf{x}-\mathbf{y}\Vert <ϵ$

取$\delta =ϵ$即可

1.7 (attainment of the maximum in the induced norm definition). Suppose that ${\mathbb{R}}^m$ and ${\mathbb{R}}^n$ are equipped with norms $\Vert \cdot {\Vert }_b$ and $\Vert \cdot {\Vert }_a$, respectively, and let $\mathbf{A}\in {\mathbb{R}}^{m\times n}$. Show that there exists $\mathbf{x}\in {\mathbb{R}}^n$ such that $\Vert \mathbf{x}{\Vert }_a\le 1$ and $\Vert \mathbf{A}\mathbf{x}{\Vert }_b=\Vert \mathbf{A}{\Vert }_{a,b}$.

解：
连续函数在紧集中可以取到最大值，所以成立

1.8 Suppose that ${\mathbb{R}}^m$ and ${\mathbb{R}}^n$ are equipped with norms $\Vert \cdot {\Vert }_b$ and $\Vert \cdot {\Vert }_a$,respectively.Show that the induced matrix norm $\Vert \cdot {\Vert }_{a,b}$ can be computed by the formula
∥ A ∥ a , b = max ⁡ x { ∥ A x ∥ b : ∥ x ∥ a = 1 } \|\mathbf{A}\|_{a,b}=\max_{\mathbf{x}}\left\{\|\mathbf{Ax}\|_b: \|\mathbf{x}\|_a=1\right\} ∥A∥a,b​=xmax​{∥Ax∥b​:∥x∥a​=1}

解：
$$\begin{array}{rl}& \Vert \mathbf{A}{\Vert }_{a,b}\\ & =\underset{\Vert \mathbf{x}{\Vert }_a\le 1}{\max }\frac{\Vert \mathbf{A}\mathbf{x}{\Vert }_b}{\Vert \mathbf{x}{\Vert }_a}\\ & =\underset{\Vert \mathbf{x}{\Vert }_a\le 1}{\max }\Vert \mathbf{A}\frac{\mathbf{x}}{\Vert \mathbf{x}{\Vert }_a}{\Vert }_b\\ & =\underset{\Vert \mathbf{y}{\Vert }_a=1}{\max }\Vert \mathbf{A}\mathbf{y}{\Vert }_b\end{array}$$

1.9 Suppose that ${\mathbb{R}}^m$ and ${\mathbb{R}}^n$ are equipped with norms $\Vert \cdot {\Vert }_b$ and $\Vert \cdot {\Vert }_a$,respectively.Show that the induced matrix norm $\Vert \cdot {\Vert }_{a,b}$ can be computed by the formula
$$\Vert \mathbf{A}{\Vert }_{a,b}=\underset{\mathbf{x}\ne 0}{\max }\frac{\Vert \mathbf{A}\mathbf{x}{\Vert }_b}{\Vert \mathbf{x}{\Vert }_a}$$

解：
$$\begin{array}{rl}& \underset{\mathbf{x}\ne 0}{\max }\frac{\Vert \mathbf{A}\mathbf{x}{\Vert }_b}{\Vert \mathbf{x}{\Vert }_a}\\ & =\underset{\mathbf{x}\ne 0}{\max }\Vert \mathbf{A}\frac{\mathbf{x}}{\Vert \mathbf{x}{\Vert }_a}{\Vert }_b\\ & =\underset{\Vert \mathbf{y}{\Vert }_a=1}{\max }\Vert \mathbf{A}\mathbf{y}{\Vert }_b\\ & =\Vert \mathbf{A}{\Vert }_{a,b}\end{array}$$

1.10 Let $\mathbf{A}\in {\mathbb{R}}^{m\times n},\mathbf{B}\in {\mathbb{R}}^{n\times k}$ and assume that ${\mathbb{R}}^m,{\mathbb{R}}^n$, and ${\mathbb{R}}^k$ are equipped with the norms $\Vert \cdot {\Vert }_c,\Vert \cdot {\Vert }_b$, and $\Vert \cdot {\Vert }_a$,respectively. Prove that
$$\Vert \mathbf{A}\mathbf{B}{\Vert }_{a,c}\le \Vert \mathbf{A}{\Vert }_{b,c}\Vert \mathbf{B}{\Vert }_{a,b}$$

解：
$$\begin{array}{rl}& \Vert \mathbf{A}\mathbf{B}{\Vert }_{a,c}\\ & =\underset{\Vert \mathbf{x}{\Vert }_a\le 1}{\max }\frac{\Vert \mathbf{A}\mathbf{B}\mathbf{x}{\Vert }_c}{\Vert \mathbf{x}{\Vert }_a}\\ & \le \underset{\Vert \mathbf{x}{\Vert }_a\le 1}{\max }\frac{\Vert \mathbf{A}{\Vert }_{b,c}\Vert \mathbf{B}\mathbf{x}{\Vert }_b}{\Vert \mathbf{x}{\Vert }_a}\\ & =\Vert \mathbf{A}{\Vert }_{b,c}\Vert \mathbf{B}{\Vert }_{a,b}\end{array}$$

1.11 Prove the formula of the $\infty$-matrix norm given in Example 1.9.

解：
$$\begin{array}{rl}& \Vert \mathbf{A}{\Vert }_{\infty }\\ & =\underset{\Vert \mathbf{x}{\Vert }_{\infty }=1}{\max }\Vert \mathbf{A}\mathbf{x}{\Vert }_{\infty }\\ & =\underset{\Vert \mathbf{x}{\Vert }_{\infty }=1}{\max }\underset{i}{\max }\left|\sum _{j=1}^n{a}_{ij}{x}_j\right|\\ & \le \underset{\Vert \mathbf{x}{\Vert }_{\infty }=1}{\max }\underset{i}{\max }\sum _{j=1}^n\left|a_{ij}{x}_j\right|\\ & \le \underset{i}{\max }\sum _{j=1}^n\left|a_{ij}\right|\end{array}$$

设${\mathbf{x}}_0=\left(x_1,\cdots ,x_n\right)$
其中$x_j=\{\begin{array}{ll}\frac{|a_{ij}|}{a_{ij}},& a_{ij}\ne 0\\ 1,& a_{ij}=0\end{array}$
$$\Vert \mathbf{A}{\mathbf{x}}_0{\Vert }_{\infty }=\underset{i}{\max }\sum _{j=1}^n\left|a_{ij}\right|$$

1.12 Let $\mathbf{A}\in {\mathbb{R}}^{m\times n}$.Prove that
(i)$\frac{1}{\sqrt{n}}\Vert \mathbf{A}{\Vert }_{\infty }\le \Vert \mathbf{A}{\Vert }_2\le \sqrt{m}\Vert \mathbf{A}{\Vert }_{\infty }$
(ii)$\frac{1}{\sqrt{m}}\Vert \mathbf{A}{\Vert }_1\le \Vert \mathbf{A}{\Vert }_2\le \sqrt{n}\Vert \mathbf{A}{\Vert }_1$

解：
(i)
利用
$$\Vert \mathbf{x}{\Vert }_{\infty }\le \Vert \mathbf{x}{\Vert }_2\le \sqrt{n}\Vert \mathbf{x}{\Vert }_{\infty }$$
（可以用 最大<=求和<=n*最大来证明）

于是
$$\begin{array}{rl}& \frac{1}{\sqrt{n}}\Vert \mathbf{A}{\Vert }_{\infty }\\ & =\frac{1}{\sqrt{n}}\underset{\mathbf{x}\ne 0}{\max }\frac{\Vert \mathbf{A}\mathbf{x}{\Vert }_{\infty }}{\Vert \mathbf{x}{\Vert }_{\infty }}\\ & \le \underset{\mathbf{x}\ne 0}{\max }\frac{\Vert \mathbf{A}\mathbf{x}{\Vert }_2}{\Vert \mathbf{x}{\Vert }_2}=\Vert \mathbf{x}{\Vert }_2\\ & \le \underset{\mathbf{x}\ne 0}{\max }\frac{\sqrt{m}\Vert \mathbf{A}\mathbf{x}{\Vert }_{\infty }}{\Vert \mathbf{x}{\Vert }_{\infty }}\\ & =\sqrt{m}\Vert \mathbf{A}{\Vert }_{\infty }\end{array}$$
(ii)
利用
$$\frac{1}{\sqrt{m}}\Vert \mathbf{x}{\Vert }_1\le \Vert \mathbf{x}{\Vert }_2\le \Vert \mathbf{x}{\Vert }_1$$
（左边用算术平均<=平方平均，右边可以数学归纳）

于是
$$\begin{array}{rl}& \frac{1}{\sqrt{m}}\Vert \mathbf{A}{\Vert }_1\\ & =\frac{1}{\sqrt{m}}\underset{\mathbf{x}\ne 0}{\max }\frac{\Vert \mathbf{A}\mathbf{x}{\Vert }_1}{\Vert \mathbf{x}{\Vert }_1}\\ & \le \underset{\mathbf{x}\ne 0}{\max }\frac{\Vert \mathbf{A}\mathbf{x}{\Vert }_2}{\Vert \mathbf{x}{\Vert }_2}=\Vert \mathbf{x}{\Vert }_2\\ & \le \underset{\mathbf{x}\ne 0}{\max }\frac{\Vert \mathbf{A}\mathbf{x}{\Vert }_1}{\frac{\Vert \mathbf{x}{\Vert }_1}{\sqrt{n}}}\\ & =\sqrt{n}\Vert \mathbf{A}{\Vert }_1\end{array}$$
1.13 Let $\mathbf{A}\in {\mathbb{R}}^{m\times n}$ Show that
(i)$\Vert \mathbf{A}\Vert =\Vert {\mathbf{A}}^T\Vert$(here $\Vert \cdot \Vert$ is the spectral norm),
(ii)$\Vert \mathbf{A}{\Vert }_F^2={\sum }_{i=1}^n{\lambda }_i\left({\mathbf{A}}^T\mathbf{A}\right)$

解：
(i)
$$\left(\begin{array}{cc}I& B\\ 0& I\end{array}\right)\left(\begin{array}{cc}0& 0\\ A& AB\end{array}\right)=\left(\begin{array}{cc}BA& 0\\ A& 0\end{array}\right)\left(\begin{array}{cc}I& B\\ 0& I\end{array}\right)$$
所以$\left(\begin{array}{cc}0& 0\\ A& AB\end{array}\right)\sim \left(\begin{array}{cc}BA& 0\\ A& 0\end{array}\right)$
因为$\left(\begin{array}{cc}0& 0\\ A& AB\end{array}\right)$的特征值为$AB$的特征值，以及几个0
$\left(\begin{array}{cc}BA& 0\\ A& 0\end{array}\right)$的特征值为$BA$的特征值，以及几个0
所以$AB$非零特征值与$BA$非零特征值相同

所以$\mathbf{A}{\mathbf{A}}^T$的非零特征值与${\mathbf{A}}^T\mathbf{A}$的非零特征值相同
$$\Vert \mathbf{A}{\Vert }_2=\sqrt{{\lambda }_1\left({\mathbf{A}}^T\mathbf{A}\right)}=\sqrt{{\lambda }_1\left(\mathbf{A}{\mathbf{A}}^T\right)}=\Vert {\mathbf{A}}^T{\Vert }_2$$
(ii)
$$\Vert \mathbf{A}{\Vert }_F^2=\sum _{i=1}^m\sum _{j=1}^n{a}_{ij}^2=tr\left({\mathbf{A}}^T\mathbf{A}\right)=\sum _{i=1}^n{\lambda }_i\left({\mathbf{A}}^T\mathbf{A}\right)$$
1.14 Let $\mathbf{A}\in {\mathbb{R}}^{n\times n}$ be a symmetric matrix.Show that
$$\underset{\mathbf{x}}{\max }\left\{{\mathbf{x}}^T\mathbf{A}\mathbf{x}:\Vert \mathbf{x}{\Vert }^2=1\right\}={\lambda }_{max}\left(\mathbf{A}\right)$$

解：
由瑞利商显然成立

1.15 Prove that a set $U\subseteq {\mathbb{R}}^n$ is closed if and only if its complement $U^c$ is open.

解：
$\Leftarrow$ 假设$U$是闭的，取$x\in U^c$,有$x\notin U$,所以$x$不是$U$的极限点。
因此存在$x$的一个邻域$N$，使得$U\cap N=\varnothing$,
于是$N\subset U^c$,即$x$是$U^c$的内点
于是$U^c$是开的

$\Rightarrow$假设$U^c$是开的，假设$x$是$U$的一个极限点，那么$x$的每一个邻域都包含$U$的点，所以$x$不是$U^c$的内点，所以$U$是开的

1.16
(i)Let { A i } i ∈ I \left\{A_i\right\}_{i\in I} {Ai​}i∈I​ be a collection of open sets where $I$ is a given index set. Show that ${\cup }_{i\in I}{A}_i$ is an open set. Show that if $I$ is finite,then ${\cup }_{i\in I}{A}_i$ is open.
(ii)Let { A i } i ∈ I \left\{A_i\right\}_{i\in I} {Ai​}i∈I​ be a collection of closed sets where $I$ is a given index set. Show that ${\cap }_{i\in I}{A}_i$ is an closed set. Show that if $I$ is finite,then ${\cap }_{i\in I}{A}_i$ is closed.

解：
(i)
因为开集的所有点都是内点
取并集后，所有点依然是内点，所以成立
(ii)
取补集，然后由(i)和1.15题，成立

1.17 Give an example of open sets $A_i,i\in I$ for which ${\cap }_{i\in I}{A}_i$ is not open

解：
∩ i = 1 ∞ ( − 1 i , 1 i ) = { 0 } \cap_{i=1}^{\infty}\left(-\frac{1}{i},\frac{1}{i}\right)=\left\{0\right\} ∩i=1∞​(−i1​,i1​)={0}

1.18 Let $A,B\subseteq {\mathbb{R}}^n$.Prove that $cl(A\cap B)\subseteq cl(A)\cap cl(B)$.Give an example in which the inclusion is proper.

解：
$A\subseteq cl(A)$
$B\subseteq cl(B)$
所以$A\cap B\subseteq cl(A)\cap cl(B)$
因为$cl(A\cap B)$是包含$A\cap B$的所有闭包中最小的
且$cl(A)\cap cl(B)$是闭集
所以
$cl(A\cap B)\subseteq cl(A)\cap cl(B)$

$A=\left(0,1\right)$
$B=\left(1,2\right)$
$cl(A\cap B)=\varnothing$
c l ( A ) ∩ c l ( B ) = { 1 } cl(A)\cap cl(B)=\left\{1\right\} cl(A)∩cl(B)={1}
$cl(A\cap B)\subset cl(A)\cap cl(B)$

1.19 Let $A,B\subseteq {\mathbb{R}}^n$.Prove that $int(A\cap B)=int(A)\cap int(B)$ and that $int(A)\cup int(B)\subseteq int(A\cup B)$.Show an example in which the latter inclusion is proper.

解：
设$x\in int(A\cap B)$,
于是$\exists r>0,B(x,r)\subset A\cap B$
所以$B(x,r)\subset A,B(x,r)\subset B\Rightarrow x\in int(A),x\in int(B)\Rightarrow x\in int(A)\cap int(B)$
所以$int(A\cap B)\subseteq int(A)\cap int(B)$

设$x\in int(A)\cap int(B)$
假设$\exists r_1>0,r_2>0,s.t.B(x,r_1)\subset A,B(x,r_2)\subset B$
取 r = min ⁡ { r 1 , r 2 } , B ( x , r ) ⊂ A , B ( x , r ) ⊂ B r=\min\left\{r_1,r_2\right\},B(x,r)\subset A,B(x,r)\subset B r=min{r1​,r2​},B(x,r)⊂A,B(x,r)⊂B
于是$B(x,r)\subset A\cap B$
所以$x\in int(A\cap B)$
所以$int(A)\cap int(B)\subseteq int(A\cap B)$

$$\Rightarrow int(A\cap B)=int(A)\cap int(B)$$

设$x\in int(A)$
则$\exists r>0,B(x,r)\subset A\subset A\cup B$
所以$x\in int(A\cup B)$
所以$int(A)\subseteq int(A\cup B)$

设$x\in int(B)$
同理$int(B)\subseteq int(A\cup B)$
$$\Rightarrow int(A)\cup int(B)\subseteq int(A\cup B)$$

设$A=(-\infty ,0],B=[0,+\infty )$
i n t ( A ) ∪ i n t ( B ) = R \ { 0 } int(A)\cup int(B)=\mathbb{R}\backslash\left\{0\right\} int(A)∪int(B)=R\{0}
$int(A\cup B)=\mathbb{R}$
所以$\Rightarrow int(A)\cup int(B)\subset int(A\cup B)$