核范数学习

设$\mathbf{A}\in {\mathbb{R}}^{m\times n}$,
核范数定义:
∥ A ∥ ∗ = ∑ i = 1 min ⁡ { m , n } σ i = ∑ i = 1 r σ i = tr ⁡ ( A T A ) \|\mathbf{A}\|_{*}=\sum_{i=1}^{\min\left\{m,n\right\}}\sigma_i=\sum_{i=1}^{r}\sigma_i=\operatorname{tr}\left(\sqrt{\mathbf{A}^T\mathbf{A}}\right) ∥A∥∗​=i=1∑min{m,n}​σi​=i=1∑r​σi​=tr(ATA ​)
其中${\sigma }_i$为奇异值,$r=\mathrm{rank}\left(\mathbf{A}\right)$

接下来证明核范数是一个范数
设奇异值 σ 1 ≥ σ 2 ≥ ⋯ ≥ σ r ≥ σ r + 1 = ⋯ = σ min ⁡ { m , n } = 0 \sigma_1\ge \sigma_2\ge\cdots\ge \sigma_r\ge \sigma_{r+1}=\cdots=\sigma_{\min\left\{m,n\right\}}= 0 σ1​≥σ2​≥⋯≥σr​≥σr+1​=⋯=σmin{m,n}​=0
非负性：
${\sigma }_i\ge 0$
当$\mathbf{A}\ne 0$时，${\sigma }_1>0$,所以$\Vert \mathbf{A}{\Vert }_{\ast }>0$
当$\mathbf{A}=0$时，${\sigma }_1=0$,所以$\Vert \mathbf{A}{\Vert }_{\ast }=0$

齐次性：
$$\Vert \lambda \mathbf{A}{\Vert }_{\ast }=\mathrm{tr}\left(\sqrt{{\lambda }^2{\mathbf{A}}^T\mathbf{A}}\right)=\left|\lambda \right|\mathrm{tr}\left(\sqrt{{\mathbf{A}}^T\mathbf{A}}\right)=\left|\lambda \right|\Vert \mathbf{A}{\Vert }_{\ast }$$

引理
设$\Vert \mathbf{x}\Vert \le 1,\Vert \mathbf{y}\Vert \le 1$
$${\mathbf{x}}^T\mathbf{Ay}\le {\sigma }_1$$
证明：
$${\mathbf{x}}^T\mathbf{Ay}\le \Vert \mathbf{x}\Vert \Vert \mathbf{Ay}\Vert \le \Vert \mathbf{A}\Vert ={\sigma }_1$$

三角不等式：
对$\mathbf{A}$进行奇异值分解和精简奇异值分解
$$\mathbf{A}=\mathbf{U\Sigma }{\mathbf{V}}^T={\mathbf{U}}_r{\mathbf{\Sigma }}_r{\mathbf{V}}_r^T$$
接下来证明${\sup }_{{\sigma }_1\left(\mathbf{Q}\right)\le 1}⟨\mathbf{Q},\mathbf{A}⟩=\Vert \mathbf{A}{\Vert }_{\ast }$
其中$⟨\mathbf{Q},\mathbf{A}⟩=\mathrm{tr}\left({\mathbf{Q}}^T\mathbf{A}\right)$

取${\mathbf{Q}}_0={\mathbf{U}}_r{\mathbf{I}}_r{\mathbf{V}}_r^T={\mathbf{U}}_r{\mathbf{V}}_r^T$
$$\begin{array}{rl}⟨{\mathbf{Q}}_0,\mathbf{A}⟩& =\mathrm{tr}\left({\mathbf{Q}}_0^T\mathbf{A}\right)\\ & =\mathrm{tr}\left({\mathbf{V}}_r{\mathbf{U}}_r^T{\mathbf{U}}_r{\mathbf{\Sigma }}_r{\mathbf{V}}_r^T\right)\\ & =\mathrm{tr}\left({\mathbf{V}}_r{\mathbf{\Sigma }}_r{\mathbf{V}}_r^T\right)\\ & =\mathrm{tr}\left({\mathbf{V}}_r^T{\mathbf{V}}_r{\mathbf{\Sigma }}_r\right)\\ & =\mathrm{tr}\left({\mathbf{\Sigma }}_r\right)\\ & =\Vert \mathbf{A}{\Vert }_{\ast }\end{array}$$
于是
$$\underset{{\sigma }_1\left(\mathbf{Q}\right)\le 1}{\sup }⟨\mathbf{Q},\mathbf{A}⟩\ge \Vert \mathbf{A}{\Vert }_{\ast }$$

sup ⁡ σ 1 ( Q ) ≤ 1 ⟨ Q , A ⟩ = sup ⁡ σ 1 ( Q ) ≤ 1 tr ⁡ ( Q T A ) = sup ⁡ σ 1 ( Q ) ≤ 1 tr ⁡ ( Q T U Σ V T ) = sup ⁡ σ 1 ( Q ) ≤ 1 tr ⁡ ( V T Q T U Σ ) = sup ⁡ σ 1 ( Q ) ≤ 1 tr ⁡ ( ( U T Q V ) T Σ ) = sup ⁡ σ 1 ( Q ) ≤ 1 ∑ i = 1 min ⁡ { m , n } σ i u i T Q v i ≤ sup ⁡ σ 1 ( Q ) ≤ 1 ∑ i = 1 min ⁡ { m , n } σ i σ 1 ( Q ) ≤ ∑ i = 1 min ⁡ { m , n } σ i = ∥ A ∥ ∗ \begin{aligned} \sup_{\sigma_1\left(\mathbf{Q}\right)\le 1}\left\langle\mathbf{Q},\mathbf{A}\right\rangle&=\sup_{\sigma_1\left(\mathbf{Q}\right)\le 1}\operatorname{tr}\left(\mathbf{Q}^T\mathbf{A}\right)\\ &=\sup_{\sigma_1\left(\mathbf{Q}\right)\le 1}\operatorname{tr}\left(\mathbf{Q}^T\mathbf{U}\mathbf{\Sigma}\mathbf{V}^T\right)\\ &=\sup_{\sigma_1\left(\mathbf{Q}\right)\le 1}\operatorname{tr}\left(\mathbf{V}^T\mathbf{Q}^T\mathbf{U}\mathbf{\Sigma}\right)\\ &=\sup_{\sigma_1\left(\mathbf{Q}\right)\le 1}\operatorname{tr}\left(\left(\mathbf{U}^T\mathbf{Q}\mathbf{V}\right)^T\mathbf{\Sigma}\right)\\ &=\sup_{\sigma_1\left(\mathbf{Q}\right)\le 1}\sum_{i=1}^{\min\left\{m,n\right\}}\sigma_i\mathbf{u}_i^T\mathbf{Q}\mathbf{v}_i\\ &\le\sup_{\sigma_1\left(\mathbf{Q}\right)\le 1}\sum_{i=1}^{\min\left\{m,n\right\}}\sigma_i\sigma_1\left({\mathbf{Q}}\right)\\ &\le\sum_{i=1}^{\min\left\{m,n\right\}}\sigma_i\\ &=\|\mathbf{A}\|_{*} \end{aligned} σ1​(Q)≤1sup​⟨Q,A⟩​=σ1​(Q)≤1sup​tr(QTA)=σ1​(Q)≤1sup​tr(QTUΣVT)=σ1​(Q)≤1sup​tr(VTQTUΣ)=σ1​(Q)≤1sup​tr((UTQV)TΣ)=σ1​(Q)≤1sup​i=1∑min{m,n}​σi​uiT​Qvi​≤σ1​(Q)≤1sup​i=1∑min{m,n}​σi​σ1​(Q)≤i=1∑min{m,n}​σi​=∥A∥∗​​
于是
$$\underset{{\sigma }_1\left(\mathbf{Q}\right)\le 1}{\sup }⟨\mathbf{Q},\mathbf{A}⟩=\Vert \mathbf{A}{\Vert }_{\ast }$$
所以
$$\begin{array}{rl}\Vert \mathbf{A}+\mathbf{B}{\Vert }_{\ast }& =\underset{{\sigma }_1\left(\mathbf{Q}\right)\le 1}{\sup }⟨\mathbf{Q},\mathbf{A}+\mathbf{B}⟩\\ & =\underset{{\sigma }_1\left(\mathbf{Q}\right)\le 1}{\sup }\left(⟨\mathbf{Q},\mathbf{A}⟩+⟨\mathbf{Q},\mathbf{B}⟩\right)\\ & \le \underset{{\sigma }_1\left(\mathbf{Q}\right)\le 1}{\sup }⟨\mathbf{Q},\mathbf{A}⟩+\underset{{\sigma }_1\left(\mathbf{Q}\right)\le 1}{\sup }⟨\mathbf{Q},\mathbf{B}⟩\\ & =\Vert \mathbf{A}{\Vert }_{\ast }+\Vert \mathbf{B}{\Vert }_{\ast }\end{array}$$

参考
https://angms.science/doc/LA/KyFanNorm.pdf