文章针对以下代码重点研究:
xformedItems = dataMat.T * U[:,:4] * Sig4.I
首先是SVD分解的公式(注意不是SVD近似公式):
M
m
⋅
n
=
U
m
⋅
m
⋅
Σ
m
⋅
n
⋅
(
V
n
⋅
n
)
T
M_{m·n}=U_{m·m}·Σ_{m·n}·{(V_{n·n})}^{T}
Mm⋅n=Um⋅m⋅Σm⋅n⋅(Vn⋅n)T
注意
Σ
Σ
Σ在python代码中返回时,是一个向量,矩阵论中是一个对角矩阵。
对角线上都是各个奇异值,其余元素都是0.
并且
Σ
Σ
Σ矩阵必定是从大到小排序过的。
其中
U
m
⋅
m
U_{m·m}
Um⋅m和
V
n
⋅
n
V_{n·n}
Vn⋅n是正交阵
正交矩阵的各行是单位向量且两两正交
正交矩阵的各列是单位向量且两两正交
对于正交阵
A
A
A(特指方阵)有以下性质:
A
T
=
A
−
1
A^T=A^{-1}
AT=A−1
如果从
U
m
⋅
m
U_{m·m}
Um⋅m中抽取k列,构成
U
m
⋅
k
U_{m·k}
Um⋅k,且
U
k
⋅
m
=
(
U
m
⋅
k
)
T
U_{k·m}=(U_{m·k})^{T}
Uk⋅m=(Um⋅k)T
如果从
V
m
⋅
m
V_{m·m}
Vm⋅m中抽取k列,构成
V
m
⋅
k
V_{m·k}
Vm⋅k,且
V
k
⋅
m
=
(
V
m
⋅
k
)
T
V_{k·m}=(V_{m·k})^{T}
Vk⋅m=(Vm⋅k)T
那么有:
U
k
⋅
m
⋅
U
m
⋅
k
=
E
k
⋅
k
①
U_{k·m}·U_{m·k}=E_{k·k}①
Uk⋅m⋅Um⋅k=Ek⋅k①
U
m
⋅
k
⋅
U
k
⋅
m
U_{m·k}·U_{k·m}
Um⋅k⋅Uk⋅m≠
E
m
⋅
m
E_{m·m}
Em⋅m
V
k
⋅
m
⋅
V
m
⋅
k
=
E
k
⋅
k
②
V_{k·m}·V_{m·k}=E_{k·k}②
Vk⋅m⋅Vm⋅k=Ek⋅k②
V
m
⋅
k
⋅
V
k
⋅
m
V_{m·k}·V_{k·m}
Vm⋅k⋅Vk⋅m≠
E
m
⋅
m
E_{m·m}
Em⋅m
上述等式的成立条件是:k≤m
注意上面①②的每个式子的顺序不能反,
接下来是SVD的近似公式:
M
m
⋅
n
≈
U
m
⋅
k
⋅
Σ
k
⋅
k
⋅
(
V
n
⋅
k
)
T
③
M_{m·n}≈U_{m·k}·Σ_{k·k}·{(V_{n·k})}^{T} ③
Mm⋅n≈Um⋅k⋅Σk⋅k⋅(Vn⋅k)T③
下面根据该式进行推导,对于③中,等式左右两边乘以
U
k
⋅
m
U_{k·m}
Uk⋅m,得到:
U
k
⋅
m
⋅
M
m
⋅
n
≈
(
U
k
⋅
m
⋅
U
m
⋅
k
)
⋅
Σ
k
⋅
k
⋅
(
V
n
⋅
k
)
T
U_{k·m}·M_{m·n}≈(U_{k·m}·U_{m·k})·Σ_{k·k}·{(V_{n·k})}^{T}
Uk⋅m⋅Mm⋅n≈(Uk⋅m⋅Um⋅k)⋅Σk⋅k⋅(Vn⋅k)T
代入式①得到:
U
k
⋅
m
⋅
M
m
⋅
n
≈
E
k
⋅
k
⋅
Σ
k
⋅
k
⋅
(
V
n
⋅
k
)
T
④
U_{k·m}·M_{m·n}≈E_{k·k}·Σ_{k·k}·{(V_{n·k})}^{T}④
Uk⋅m⋅Mm⋅n≈Ek⋅k⋅Σk⋅k⋅(Vn⋅k)T④
两边乘以
(
Σ
k
⋅
k
)
−
1
(Σ_{k·k})^{-1}
(Σk⋅k)−1,得到:
(
Σ
k
⋅
k
)
−
1
⋅
U
k
⋅
m
⋅
M
m
⋅
n
≈
(
V
n
⋅
k
)
T
⑤
{(Σ_{k·k}})^{-1}·U_{k·m}·M_{m·n}≈{(V_{n·k})}^{T}⑤
(Σk⋅k)−1⋅Uk⋅m⋅Mm⋅n≈(Vn⋅k)T⑤
等式两边进行转置操作:
V
n
⋅
k
≈
(
(
Σ
k
⋅
k
)
−
1
⋅
U
k
⋅
m
⋅
M
m
⋅
n
)
T
{V_{n·k}}≈({(Σ_{k·k}})^{-1}·U_{k·m}·M_{m·n})^{T}
Vn⋅k≈((Σk⋅k)−1⋅Uk⋅m⋅Mm⋅n)T
对等式右边进行展开:
V
n
⋅
k
≈
(
M
m
⋅
n
)
T
⋅
U
m
⋅
k
⋅
(
(
Σ
k
⋅
k
)
−
1
)
T
⑥
{V_{n·k}}≈({M_{m·n})^{T}·U_{m·k}·((Σ_{k·k}})^{-1})^{T}⑥
Vn⋅k≈(Mm⋅n)T⋅Um⋅k⋅((Σk⋅k)−1)T⑥
∵
(
Σ
k
⋅
k
)
(Σ_{k·k})
(Σk⋅k)是对角阵,
∴
(
Σ
k
⋅
k
)
−
1
(Σ_{k·k})^{-1}
(Σk⋅k)−1也是对角阵
∴
(
Σ
k
⋅
k
)
−
1
(Σ_{k·k})^{-1}
(Σk⋅k)−1的转置矩阵等于
(
Σ
k
⋅
k
)
−
1
⑦
(Σ_{k·k})^{-1}⑦
(Σk⋅k)−1⑦
将⑦中关系,代入⑥,得到
V n ⋅ k ≈ ( M m ⋅ n ) T ⋅ U m ⋅ k ⋅ ( Σ k ⋅ k ) − 1 ⑧ {V_{n·k}}≈({M_{m·n})^{T}·U_{m·k}·(Σ_{k·k}})^{-1}⑧ Vn⋅k≈(Mm⋅n)T⋅Um⋅k⋅(Σk⋅k)−1⑧
对比⑧和书上的这句代码进行比较:
xformedItems = dataMat.T * U[:,:4] * Sig4.I
我们会发现是一模一样的。
所以上述推导是这句代码的理论依据。
借助的在线公式编辑器链接为:
http://www.sciweavers.org/free-online-latex-equation-editor