设
A
∈
R
m
×
n
\mathbf{A}\in\mathbb{R}^{m\times n}
A∈Rm×n,
核范数定义:
∥
A
∥
∗
=
∑
i
=
1
min
{
m
,
n
}
σ
i
=
∑
i
=
1
r
σ
i
=
tr
(
A
T
A
)
\|\mathbf{A}\|_{*}=\sum_{i=1}^{\min\left\{m,n\right\}}\sigma_i=\sum_{i=1}^{r}\sigma_i=\operatorname{tr}\left(\sqrt{\mathbf{A}^T\mathbf{A}}\right)
∥A∥∗=i=1∑min{m,n}σi=i=1∑rσi=tr(ATA)
其中
σ
i
\sigma_i
σi为奇异值,
r
=
rank
(
A
)
r=\operatorname{rank}\left(\mathbf{A}\right)
r=rank(A)
接下来证明核范数是一个范数
设奇异值
σ
1
≥
σ
2
≥
⋯
≥
σ
r
≥
σ
r
+
1
=
⋯
=
σ
min
{
m
,
n
}
=
0
\sigma_1\ge \sigma_2\ge\cdots\ge \sigma_r\ge \sigma_{r+1}=\cdots=\sigma_{\min\left\{m,n\right\}}= 0
σ1≥σ2≥⋯≥σr≥σr+1=⋯=σmin{m,n}=0
非负性:
σ
i
≥
0
\sigma_i\ge 0
σi≥0
当
A
≠
0
\mathbf{A}\neq 0
A=0时,
σ
1
>
0
\sigma_1>0
σ1>0,所以
∥
A
∥
∗
>
0
\|\mathbf{A}\|_*>0
∥A∥∗>0
当
A
=
0
\mathbf{A}=0
A=0时,
σ
1
=
0
\sigma_1=0
σ1=0,所以
∥
A
∥
∗
=
0
\|\mathbf{A}\|_*=0
∥A∥∗=0
齐次性:
∥
λ
A
∥
∗
=
tr
(
λ
2
A
T
A
)
=
∣
λ
∣
tr
(
A
T
A
)
=
∣
λ
∣
∥
A
∥
∗
\|\lambda\mathbf{A}\|_*=\operatorname{tr}\left(\sqrt{\lambda^2\mathbf{A}^T\mathbf{A}}\right)=\left|\lambda\right|\operatorname{tr}\left(\sqrt{\mathbf{A}^T\mathbf{A}}\right)=\left|\lambda\right|\|\mathbf{A}\|_*
∥λA∥∗=tr(λ2ATA)=∣λ∣tr(ATA)=∣λ∣∥A∥∗
引理
设
∥
x
∥
≤
1
,
∥
y
∥
≤
1
\|\mathbf{x}\|\le1,\|\mathbf{y}\|\le1
∥x∥≤1,∥y∥≤1
x
T
A
y
≤
σ
1
\mathbf{x}^T\mathbf{Ay}\le\sigma_1
xTAy≤σ1
证明:
x
T
A
y
≤
∥
x
∥
∥
A
y
∥
≤
∥
A
∥
=
σ
1
\mathbf{x}^T\mathbf{Ay}\le\|\mathbf{x}\|\|\mathbf{Ay}\|\le\|\mathbf{A}\|=\sigma_1
xTAy≤∥x∥∥Ay∥≤∥A∥=σ1
三角不等式:
对
A
\mathbf{A}
A进行奇异值分解和精简奇异值分解
A
=
U
Σ
V
T
=
U
r
Σ
r
V
r
T
\mathbf{A}=\mathbf{U}\mathbf{\Sigma}\mathbf{V}^T=\mathbf{U}_r\mathbf{\Sigma}_r\mathbf{V}_r^T
A=UΣVT=UrΣrVrT
接下来证明
sup
σ
1
(
Q
)
≤
1
⟨
Q
,
A
⟩
=
∥
A
∥
∗
\sup_{\sigma_1\left(\mathbf{Q}\right)\le 1}\left\langle\mathbf{Q},\mathbf{A}\right\rangle=\|\mathbf{A}\|_*
supσ1(Q)≤1⟨Q,A⟩=∥A∥∗
其中
⟨
Q
,
A
⟩
=
tr
(
Q
T
A
)
\left\langle\mathbf{Q},\mathbf{A}\right\rangle=\operatorname{tr}\left(\mathbf{Q}^T\mathbf{A}\right)
⟨Q,A⟩=tr(QTA)
取
Q
0
=
U
r
I
r
V
r
T
=
U
r
V
r
T
\mathbf{Q}_0=\mathbf{U}_r\mathbf{I}_r\mathbf{V}_r^T=\mathbf{U}_r\mathbf{V}_r^T
Q0=UrIrVrT=UrVrT
⟨
Q
0
,
A
⟩
=
tr
(
Q
0
T
A
)
=
tr
(
V
r
U
r
T
U
r
Σ
r
V
r
T
)
=
tr
(
V
r
Σ
r
V
r
T
)
=
tr
(
V
r
T
V
r
Σ
r
)
=
tr
(
Σ
r
)
=
∥
A
∥
∗
\begin{aligned} \left\langle\mathbf{Q}_0,\mathbf{A}\right\rangle&=\operatorname{tr}\left(\mathbf{Q}_0^T\mathbf{A}\right)\\ &=\operatorname{tr}\left(\mathbf{V}_r\mathbf{U}_r^T\mathbf{U}_r\mathbf{\Sigma}_r\mathbf{V}_r^T\right)\\ &=\operatorname{tr}\left(\mathbf{V}_r\mathbf{\Sigma}_r\mathbf{V}_r^T\right)\\ &=\operatorname{tr}\left(\mathbf{V}_r^T\mathbf{V}_r\mathbf{\Sigma}_r\right)\\ &=\operatorname{tr}\left(\mathbf{\Sigma}_r\right)\\ &=\|\mathbf{A}\|_* \end{aligned}
⟨Q0,A⟩=tr(Q0TA)=tr(VrUrTUrΣrVrT)=tr(VrΣrVrT)=tr(VrTVrΣr)=tr(Σr)=∥A∥∗
于是
sup
σ
1
(
Q
)
≤
1
⟨
Q
,
A
⟩
≥
∥
A
∥
∗
\sup_{\sigma_1\left(\mathbf{Q}\right)\le 1}\left\langle\mathbf{Q},\mathbf{A}\right\rangle\ge\|\mathbf{A}\|_*
σ1(Q)≤1sup⟨Q,A⟩≥∥A∥∗
sup
σ
1
(
Q
)
≤
1
⟨
Q
,
A
⟩
=
sup
σ
1
(
Q
)
≤
1
tr
(
Q
T
A
)
=
sup
σ
1
(
Q
)
≤
1
tr
(
Q
T
U
Σ
V
T
)
=
sup
σ
1
(
Q
)
≤
1
tr
(
V
T
Q
T
U
Σ
)
=
sup
σ
1
(
Q
)
≤
1
tr
(
(
U
T
Q
V
)
T
Σ
)
=
sup
σ
1
(
Q
)
≤
1
∑
i
=
1
min
{
m
,
n
}
σ
i
u
i
T
Q
v
i
≤
sup
σ
1
(
Q
)
≤
1
∑
i
=
1
min
{
m
,
n
}
σ
i
σ
1
(
Q
)
≤
∑
i
=
1
min
{
m
,
n
}
σ
i
=
∥
A
∥
∗
\begin{aligned} \sup_{\sigma_1\left(\mathbf{Q}\right)\le 1}\left\langle\mathbf{Q},\mathbf{A}\right\rangle&=\sup_{\sigma_1\left(\mathbf{Q}\right)\le 1}\operatorname{tr}\left(\mathbf{Q}^T\mathbf{A}\right)\\ &=\sup_{\sigma_1\left(\mathbf{Q}\right)\le 1}\operatorname{tr}\left(\mathbf{Q}^T\mathbf{U}\mathbf{\Sigma}\mathbf{V}^T\right)\\ &=\sup_{\sigma_1\left(\mathbf{Q}\right)\le 1}\operatorname{tr}\left(\mathbf{V}^T\mathbf{Q}^T\mathbf{U}\mathbf{\Sigma}\right)\\ &=\sup_{\sigma_1\left(\mathbf{Q}\right)\le 1}\operatorname{tr}\left(\left(\mathbf{U}^T\mathbf{Q}\mathbf{V}\right)^T\mathbf{\Sigma}\right)\\ &=\sup_{\sigma_1\left(\mathbf{Q}\right)\le 1}\sum_{i=1}^{\min\left\{m,n\right\}}\sigma_i\mathbf{u}_i^T\mathbf{Q}\mathbf{v}_i\\ &\le\sup_{\sigma_1\left(\mathbf{Q}\right)\le 1}\sum_{i=1}^{\min\left\{m,n\right\}}\sigma_i\sigma_1\left({\mathbf{Q}}\right)\\ &\le\sum_{i=1}^{\min\left\{m,n\right\}}\sigma_i\\ &=\|\mathbf{A}\|_{*} \end{aligned}
σ1(Q)≤1sup⟨Q,A⟩=σ1(Q)≤1suptr(QTA)=σ1(Q)≤1suptr(QTUΣVT)=σ1(Q)≤1suptr(VTQTUΣ)=σ1(Q)≤1suptr((UTQV)TΣ)=σ1(Q)≤1supi=1∑min{m,n}σiuiTQvi≤σ1(Q)≤1supi=1∑min{m,n}σiσ1(Q)≤i=1∑min{m,n}σi=∥A∥∗
于是
sup
σ
1
(
Q
)
≤
1
⟨
Q
,
A
⟩
=
∥
A
∥
∗
\sup_{\sigma_1\left(\mathbf{Q}\right)\le 1}\left\langle\mathbf{Q},\mathbf{A}\right\rangle=\|\mathbf{A}\|_*
σ1(Q)≤1sup⟨Q,A⟩=∥A∥∗
所以
∥
A
+
B
∥
∗
=
sup
σ
1
(
Q
)
≤
1
⟨
Q
,
A
+
B
⟩
=
sup
σ
1
(
Q
)
≤
1
(
⟨
Q
,
A
⟩
+
⟨
Q
,
B
⟩
)
≤
sup
σ
1
(
Q
)
≤
1
⟨
Q
,
A
⟩
+
sup
σ
1
(
Q
)
≤
1
⟨
Q
,
B
⟩
=
∥
A
∥
∗
+
∥
B
∥
∗
\begin{aligned} \|\mathbf{A}+\mathbf{B}\|_*&=\sup_{\sigma_1\left(\mathbf{Q}\right)\le 1}\left\langle\mathbf{Q},\mathbf{A}+\mathbf{B}\right\rangle\\ &=\sup_{\sigma_1\left(\mathbf{Q}\right)\le 1}\left(\left\langle\mathbf{Q},\mathbf{A}\right\rangle+\left\langle\mathbf{Q},\mathbf{B}\right\rangle\right)\\ &\le \sup_{\sigma_1\left(\mathbf{Q}\right)\le 1}\left\langle\mathbf{Q},\mathbf{A}\right\rangle+\sup_{\sigma_1\left(\mathbf{Q}\right)\le 1}\left\langle\mathbf{Q},\mathbf{B}\right\rangle\\ &=\|\mathbf{A}\|_*+\|\mathbf{B}\|_* \end{aligned}
∥A+B∥∗=σ1(Q)≤1sup⟨Q,A+B⟩=σ1(Q)≤1sup(⟨Q,A⟩+⟨Q,B⟩)≤σ1(Q)≤1sup⟨Q,A⟩+σ1(Q)≤1sup⟨Q,B⟩=∥A∥∗+∥B∥∗