机器学习-白板推导 P5_2
降维基础知识
X
=
[
x
1
x
2
.
.
.
x
N
]
T
=
[
x
1
T
x
2
T
⋮
x
N
T
]
=
[
x
11
x
12
.
.
.
x
1
p
x
21
x
22
.
.
.
x
2
p
⋮
⋮
⋱
⋮
x
N
1
x
N
2
.
.
.
x
N
p
]
N
∗
p
X=\begin{bmatrix} x_1 & x_2 &...& x_N \end{bmatrix}^T=\begin{bmatrix} x_1^T \\ x_2^T \\\vdots\\ x_N^T \end{bmatrix} = \begin{bmatrix} x_{11} & x_{12} &...& x_{1p} \\ x_{21} & x_{22} &...& x_{2p} \\ \vdots & \vdots & \ddots & \vdots \\ x_{N1} & x_{N2} &...& x_{Np} \\ \end{bmatrix}_{N*p}
X=[x1x2...xN]T=⎣⎢⎢⎢⎡x1Tx2T⋮xNT⎦⎥⎥⎥⎤=⎣⎢⎢⎢⎡x11x21⋮xN1x12x22⋮xN2......⋱...x1px2p⋮xNp⎦⎥⎥⎥⎤N∗p
1
N
=
[
1
1
⋮
1
]
1_N= \begin{bmatrix} 1 \\ 1 \\\vdots\\ 1 \end{bmatrix}
1N=⎣⎢⎢⎢⎡11⋮1⎦⎥⎥⎥⎤
x
i
∈
R
p
,
y
i
∈
R
,
i
=
1
,
2...
N
x_i \in R^p, y_i \in R, i=1,2...N
xi∈Rp,yi∈R,i=1,2...N
S a m p l e M e a n : X ‾ p ∗ 1 = 1 N ∑ i = 1 N x i Sample Mean: \overline{X}_{p*1}=\frac{1}{N}\sum_{i=1}^Nx_i SampleMean:Xp∗1=N1∑i=1Nxi
S a m p l e C o v a r i a n c e : S p ∗ p = 1 N ∑ i = 1 N ( x i − X ‾ ) ( x i − X ‾ ) T Sample Covariance:S_{p*p}=\frac{1}{N}\sum_{i=1}^N(x_i-\overline{X})(x_i-\overline{X})^T SampleCovariance:Sp∗p=N1∑i=1N(xi−X)(xi−X)T
X
‾
=
1
N
∑
i
=
1
N
x
i
=
1
N
[
x
1
x
2
.
.
.
x
N
]
[
1
1
⋮
1
]
=
1
N
X
T
1
N
\overline{X}=\frac{1}{N}\sum_{i=1}^Nx_i=\frac{1}{N}\begin{bmatrix} x_1 & x_2 &...& x_N \end{bmatrix} \begin{bmatrix} 1 \\ 1 \\\vdots\\ 1 \end{bmatrix}=\frac{1}{N}X^T1_N
X=N1i=1∑Nxi=N1[x1x2...xN]⎣⎢⎢⎢⎡11⋮1⎦⎥⎥⎥⎤=N1XT1N
S
=
1
N
∑
i
=
1
N
(
x
i
−
X
‾
)
(
x
i
−
X
‾
)
T
=
1
N
[
x
1
−
X
‾
x
2
−
X
‾
.
.
.
x
N
−
X
‾
]
[
(
x
1
−
X
‾
)
T
(
x
2
−
X
‾
)
T
⋮
(
x
N
−
X
‾
)
T
]
=
1
N
(
X
T
−
X
‾
1
N
T
)
(
X
T
−
X
‾
1
N
T
)
T
=
1
N
(
X
T
−
1
N
X
T
1
N
1
N
T
)
(
X
T
−
1
N
X
T
1
N
1
N
T
)
T
=
1
N
X
T
(
I
N
−
1
N
1
N
1
N
T
)
(
I
N
−
1
N
1
N
1
N
T
)
T
X
\begin{aligned} S &=\frac{1}{N}\sum_{i=1}^N(x_i-\overline{X})(x_i-\overline{X})^T \\ &=\frac{1}{N}\begin{bmatrix} x_1 -\overline{X} & x_2-\overline{X} &...& x_N -\overline{X}\end{bmatrix} \begin{bmatrix} (x_1-\overline{X})^T \\ (x_2-\overline{X})^T\\\vdots\\ (x_N-\overline{X})^T\end{bmatrix} \\ &= \frac{1}{N}(X^T-\overline{X}{1_N}^T)(X^T-\overline{X}{1_N}^T)^T\\ &=\frac{1}{N}(X^T-\frac{1}{N}X^T1_N{1_N}^T)(X^T-\frac{1}{N}X^T1_N{1_N}^T)^T\\ &=\frac{1}{N}X^T(I_N-\frac{1}{N}1_N{1_N}^T)(I_N-\frac{1}{N}1_N{1_N}^T)^TX\\ \end{aligned}
S=N1i=1∑N(xi−X)(xi−X)T=N1[x1−Xx2−X...xN−X]⎣⎢⎢⎢⎡(x1−X)T(x2−X)T⋮(xN−X)T⎦⎥⎥⎥⎤=N1(XT−X1NT)(XT−X1NT)T=N1(XT−N1XT1N1NT)(XT−N1XT1N1NT)T=N1XT(IN−N11N1NT)(IN−N11N1NT)TX
定义:
H
N
=
I
N
−
1
N
1
N
1
N
T
→
c
e
n
t
e
r
i
n
g
  
m
a
t
r
i
x
H_N=I_N-\frac{1}{N}1_N{1_N}^T \rightarrow centering \; matrix
HN=IN−N11N1NT→centeringmatrix
H
N
H_N
HN每个样本减去均值，会使图像向中心移动
S
=
1
N
X
T
H
H
T
X
\begin{aligned} S &= \frac{1}{N}X^THH^TX \end{aligned}
S=N1XTHHTX
H N = I N − 1 N 1 N 1 N T H_N=I_N-\frac{1}{N}1_N{1_N}^T HN=IN−N11N1NT
H N T = I N − 1 N 1 N 1 N T = H N H_N^T=I_N-\frac{1}{N}1_N{1_N}^T = H_N HNT=IN−N11N1NT=HN
H 2 = H . H = ( I N − 1 N 1 N 1 N T ) ( I N − 1 N 1 N 1 N T ) = I N − 2 N 1 N 1 N T + 1 N 2 1 N 1 N T 1 N 1 N T = I N − 1 N 1 N 1 N T = H N H^2=H.H=(I_N-\frac{1}{N}1_N{1_N}^T) (I_N-\frac{1}{N}1_N{1_N}^T) =I_N-\frac{2}{N}1_N{1_N}^T+\frac{1}{N^2}1_N{1_N}^T1_N{1_N}^T=I_N-\frac{1}{N}1_N{1_N}^T = H_N H2=H.H=(IN−N11N1NT)(IN−N11N1NT)=IN−N21N1NT+N211N1NT1N1NT=IN−N11N1NT=HN
H N = H H^N=H HN=H
所以：
S
=
1
N
X
T
H
H
T
X
=
1
N
X
T
H
X
\begin{aligned} S &= \frac{1}{N}X^THH^TX = \frac{1}{N}X^THX \end{aligned}
S=N1XTHHTX=N1XTHX
138
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
