Min-max theorem

瑞利商

设$\mathbf{A}$是厄尔米特矩阵$(\mathbf{A}={\mathbf{A}}^H$)
R A ( x ) : C n \ { 0 } → R R_{\mathbf{A}}\left(\mathbf{x}\right):\mathbb{C}^n\backslash\left\{\mathbf{0}\right\}\to\mathbb{R} RA​(x):Cn\{0}→R定义为
$$R_{\mathbf{A}}=\frac{\left(\mathbf{A}\mathbf{x},\mathbf{x}\right)}{\left(\mathbf{x},\mathbf{x}\right)}$$

Min-max theorem

设$\mathbf{A}$是厄尔米特矩阵
设$\mathbf{A}$的特征值为${\lambda }_1\le {\lambda }_2\le \cdots \le {\lambda }_n$
则
λ k = min ⁡ U { max ⁡ x { R A ( x ) ∣ x ∈ U , x ≠ 0 } ∣ dim ⁡ ( U ) = k } \lambda_k=\min_{\mathbf{U}}\left\{\max_{\mathbf{x}}\left\{R_{\mathbf{A}}\left(\mathbf{x}\right)|\mathbf{x}\in\mathbf{U},\mathbf{x}\neq 0\right\}|\operatorname{dim}\left(\mathbf{U}\right)=k\right\} λk​=Umin​{xmax​{RA​(x)∣x∈U,x​=0}∣dim(U)=k}
λ k = max ⁡ U { min ⁡ x { R A ( x ) ∣ x ∈ U , x ≠ 0 } ∣ dim ⁡ ( U ) = n − k + 1 } \lambda_k=\max_{\mathbf{U}}\left\{\min_{\mathbf{x}}\left\{R_{\mathbf{A}}\left(\mathbf{x}\right)|\mathbf{x}\in\mathbf{U},\mathbf{x}\neq 0\right\}|\operatorname{dim}\left(\mathbf{U}\right)=n-k+1\right\} λk​=Umax​{xmin​{RA​(x)∣x∈U,x​=0}∣dim(U)=n−k+1}

证明：
因为$\mathbf{A}$是厄尔米特矩阵，所以可以对角化
设$\mathbf{A}$两两正交的单位特征向量为${\mathbf{u}}_1,\cdots ,{\mathbf{u}}_n$
即$\mathbf{A}{\mathbf{u}}_i={\lambda }_i{\mathbf{u}}_i,\left({\mathbf{u}}_i,{\mathbf{u}}_j\right)=0,\left({\mathbf{u}}_i{\mathbf{u}}_i\right)=1$其中$i\ne j$

先证明第一个等式
设$\dim \left(\mathbf{U}\right)=k$
dim ⁡ ( U ) + dim ⁡ ( span ⁡ { u k , ⋯   , u n } ) = k + n − k + 1 = n + 1 > n \operatorname{dim}\left(\mathbf{U}\right)+\operatorname{dim}\left(\operatorname{span}\left\{\mathbf{u}_k,\cdots,\mathbf{u}_n\right\}\right)=k+n-k+1=n+1>n dim(U)+dim(span{uk​,⋯,un​})=k+n−k+1=n+1>n
所以
U ∩ span ⁡ { u k , ⋯   , u n } ≠ { 0 } \mathbf{U}\cap \operatorname{span}\left\{\mathbf{u}_k,\cdots,\mathbf{u}_n\right\}\neq\left\{\mathbf{0}\right\} U∩span{uk​,⋯,un​}​={0}
所以存在$\mathbf{v}\ne 0$
使得 v ∈ U ∩ span ⁡ { u k , ⋯   , u n } \mathbf{v}\in \mathbf{U}\cap \operatorname{span}\left\{\mathbf{u}_k,\cdots,\mathbf{u}_n\right\} v∈U∩span{uk​,⋯,un​}

设$\mathbf{v}={\sum }_{i=k}^n{\alpha }_i{\mathbf{u}}_i$
则
$$R_{\mathbf{A}}\left(\mathbf{v}\right)=\frac{{\sum }_{i=k}^n{\lambda }_i{\alpha }_i^2}{{\sum }_{i=k}^n{\alpha }_i^2}\ge {\lambda }_k$$
所以
max ⁡ { R A ( x ) ∣ x ∈ U } ≥ R A ( v ) ≥ λ k \max \left\{R_{\mathbf{A}}\left(\mathbf{x}\right)|\mathbf{x}\in\mathbf{U}\right\}\ge R_{\mathbf{A}}\left(\mathbf{v}\right)\ge \lambda_k max{RA​(x)∣x∈U}≥RA​(v)≥λk​

因为对于任意的$\mathbf{U}$都是成立的
所以
min ⁡ U { max ⁡ x { R A ( x ) ∣ x ∈ U , x ≠ 0 } ∣ dim ⁡ ( U ) = k } ≥ λ k \min_{\mathbf{U}}\left\{\max_{\mathbf{x}}\left\{R_{\mathbf{A}}\left(\mathbf{x}\right)|\mathbf{x}\in\mathbf{U},\mathbf{x}\neq 0\right\}|\operatorname{dim}\left(\mathbf{U}\right)=k\right\}\ge \lambda_k Umin​{xmax​{RA​(x)∣x∈U,x​=0}∣dim(U)=k}≥λk​

接着证明反过来的
设 V = span ⁡ { u 1 , ⋯   , u k } \mathbf{V}=\operatorname{span}\left\{\mathbf{u}_1,\cdots,\mathbf{u}_k\right\} V=span{u1​,⋯,uk​}
则
max ⁡ { R A ( x ) ∣ x ∈ V } ≤ λ k \max \left\{R_{\mathbf{A}}\left(\mathbf{x}\right)|\mathbf{x}\in\mathbf{V}\right\}\le\lambda_k max{RA​(x)∣x∈V}≤λk​
于是
min ⁡ U { max ⁡ x { R A ( x ) ∣ x ∈ U , x ≠ 0 } ∣ dim ⁡ ( U ) = k } ≤ max ⁡ { R A ( x ) ∣ x ∈ V } ≤ λ k \min_{\mathbf{U}}\left\{\max_{\mathbf{x}}\left\{R_{\mathbf{A}}\left(\mathbf{x}\right)|\mathbf{x}\in\mathbf{U},\mathbf{x}\neq 0\right\}|\operatorname{dim}\left(\mathbf{U}\right)=k\right\}\le \max \left\{R_{\mathbf{A}}\left(\mathbf{x}\right)|\mathbf{x}\in\mathbf{V}\right\}\le\lambda_k Umin​{xmax​{RA​(x)∣x∈U,x​=0}∣dim(U)=k}≤max{RA​(x)∣x∈V}≤λk​
所以
λ k = min ⁡ U { max ⁡ x { R A ( x ) ∣ x ∈ U , x ≠ 0 } ∣ dim ⁡ ( U ) = k } \lambda_k=\min_{\mathbf{U}}\left\{\max_{\mathbf{x}}\left\{R_{\mathbf{A}}\left(\mathbf{x}\right)|\mathbf{x}\in\mathbf{U},\mathbf{x}\neq 0\right\}|\operatorname{dim}\left(\mathbf{U}\right)=k\right\} λk​=Umin​{xmax​{RA​(x)∣x∈U,x​=0}∣dim(U)=k}

接着证明第二个等式
设$\dim \left(\mathbf{U}\right)=n-k+1$
dim ⁡ ( U ) + dim ⁡ ( span ⁡ { u 1 , ⋯   , u k } ) = n − k + 1 + k = n + 1 > n \operatorname{dim}\left(\mathbf{U}\right)+\operatorname{dim}\left(\operatorname{span}\left\{\mathbf{u}_1,\cdots,\mathbf{u}_k\right\}\right)=n-k+1+k=n+1>n dim(U)+dim(span{u1​,⋯,uk​})=n−k+1+k=n+1>n
所以
U ∩ span ⁡ { u 1 , ⋯   , u k } ≠ { 0 } \mathbf{U}\cap \operatorname{span}\left\{\mathbf{u}_1,\cdots,\mathbf{u}_k\right\}\neq\left\{\mathbf{0}\right\} U∩span{u1​,⋯,uk​}​={0}
所以存在$\mathbf{v}\ne 0$
使得 v ∈ U ∩ span ⁡ { u k , ⋯   , u n } \mathbf{v}\in \mathbf{U}\cap \operatorname{span}\left\{\mathbf{u}_k,\cdots,\mathbf{u}_n\right\} v∈U∩span{uk​,⋯,un​}
设$\mathbf{v}={\sum }_{i=1}^k{\alpha }_i{\mathbf{u}}_i$
则
$$R_{\mathbf{A}}\left(\mathbf{v}\right)=\frac{{\sum }_{i=1}^k{\lambda }_i{\alpha }_i^2}{{\sum }_{i=1}^k{\alpha }_i^2}\le {\lambda }_k$$
所以
min ⁡ { R A ( x ) ∣ x ∈ U } ≤ R A ( v ) ≤ λ k \min \left\{R_{\mathbf{A}}\left(\mathbf{x}\right)|\mathbf{x}\in\mathbf{U}\right\}\le R_{\mathbf{A}}\left(\mathbf{v}\right)\le \lambda_k min{RA​(x)∣x∈U}≤RA​(v)≤λk​
因为对于任意的$\mathbf{U}$都是成立的
所以
max ⁡ U { min ⁡ x { R A ( x ) ∣ x ∈ U , x ≠ 0 } ∣ dim ⁡ ( U ) = n − k + 1 } ≤ λ k \max_{\mathbf{U}}\left\{\min_{\mathbf{x}}\left\{R_{\mathbf{A}}\left(\mathbf{x}\right)|\mathbf{x}\in\mathbf{U},\mathbf{x}\neq 0\right\}|\operatorname{dim}\left(\mathbf{U}\right)=n-k+1\right\}\le \lambda_k Umax​{xmin​{RA​(x)∣x∈U,x​=0}∣dim(U)=n−k+1}≤λk​

接着证明反过来的
设 V = span ⁡ { u k , ⋯   , u n } \mathbf{V}=\operatorname{span}\left\{\mathbf{u}_k,\cdots,\mathbf{u}_n\right\} V=span{uk​,⋯,un​}
则
min ⁡ { R A ( x ) ∣ x ∈ V } ≥ λ k \min \left\{R_{\mathbf{A}}\left(\mathbf{x}\right)|\mathbf{x}\in\mathbf{V}\right\}\ge\lambda_k min{RA​(x)∣x∈V}≥λk​
于是
max ⁡ U { min ⁡ x { R A ( x ) ∣ x ∈ U , x ≠ 0 } ∣ dim ⁡ ( U ) = n − k + 1 } ≥ min ⁡ { R A ( x ) ∣ x ∈ V } ≥ λ k \max_{\mathbf{U}}\left\{\min_{\mathbf{x}}\left\{R_{\mathbf{A}}\left(\mathbf{x}\right)|\mathbf{x}\in\mathbf{U},\mathbf{x}\neq 0\right\}|\operatorname{dim}\left(\mathbf{U}\right)=n-k+1\right\}\ge \min \left\{R_{\mathbf{A}}\left(\mathbf{x}\right)|\mathbf{x}\in\mathbf{V}\right\}\ge\lambda_k Umax​{xmin​{RA​(x)∣x∈U,x​=0}∣dim(U)=n−k+1}≥min{RA​(x)∣x∈V}≥λk​
所以
λ k = max ⁡ U { min ⁡ x { R A ( x ) ∣ x ∈ U , x ≠ 0 } ∣ dim ⁡ ( U ) = n − k + 1 } \lambda_k=\max_{\mathbf{U}}\left\{\min_{\mathbf{x}}\left\{R_{\mathbf{A}}\left(\mathbf{x}\right)|\mathbf{x}\in\mathbf{U},\mathbf{x}\neq 0\right\}|\operatorname{dim}\left(\mathbf{U}\right)=n-k+1\right\} λk​=Umax​{xmin​{RA​(x)∣x∈U,x​=0}∣dim(U)=n−k+1}

奇异值版

设${\sigma }_k$是$\mathbf{A}$的奇异值
则
$${\sigma }_k=\underset{\mathbf{S}:\dim \left(\mathbf{S}\right)=k}{\min }\underset{\mathbf{x}\in \mathbf{S},\Vert \mathbf{x}\Vert =1}{\max }\Vert \mathbf{A}\mathbf{x}\Vert$$
$${\sigma }_k=\underset{\mathbf{S}:\dim \left(\mathbf{S}\right)=n-k+1}{\min }\underset{\mathbf{x}\in \mathbf{S},\Vert \mathbf{x}\Vert =1}{\max }\Vert \mathbf{A}\mathbf{x}\Vert$$

证明：
因为
$$\sigma \left(\mathbf{A}\right)=\sqrt{\lambda \left({\mathbf{A}}^H\mathbf{A}\right)}$$
并且${\mathbf{A}}^H\mathbf{A}$是一个厄尔米特矩阵，
然后证明起来就类似了，只不过要开根号