机器学习导论（张志华）：主元分析

前言

这个笔记是北大那位老师课程的学习笔记，讲的概念浅显易懂，非常有利于我们掌握基本的概念，从而掌握相关的技术。

basic concepts

$$exp(-t{z}^{\frac{1}{2}})=\int exp(-tuz)dF(u)$$
$$z=||x|{|}^2$$
$$exp(-t||x||),exp(-t||x||).$$
The product of P.D is P.D
eul distance transformed into another space to get the distance.
$$||\varphi (x)-\varphi (y)|{|}_2^2$$
Part2 unsuperrised learning
CB dimensionlity reduction.

PCA(Principal Component Analysis)

Population PCA
Def. if $$\overline{x}\subset R^{p}isarandomvector,withmean:uandcovariancematrix\sigma$$
then the PCA is
$$\overline{x}->\overline{y}=U^t(x-u)$$
when U is orthgonal.

Spectral Decompistion

Thm,
$$Ifx->N(\mu ,\sigma )$$ Then,$$yN(0,n)$$
(2)$$E(y_0)=0,$$
(3)$$Cov(Y_m,Y_i)=0fori!=j$$
(4)$$yisaorthangonaltransformxisuncorrelationbutotsqure.$$
(5)$$Var(Y_i)={\sigma }_i$$

Sample Principal Component

$$LetX=[{\overline{x}}_1...{\overline{x}}_n{]}^{T}bean\ast p$$

sample data matrix

$$\overline{x}=\frac{1}{n}\sum _{x=1}^n{\overline{x}}_i,$$
$$S=\frac{1}{n}{X}^{T}HX$$
$$H:I_n=\frac{1}{n}{I}_n{I}_n$$
reduce the data to k-dimension ,you get the first k element.
keep most information,PCA.suppos.

SVD

$$U=eigenvectorof(A{A}^T)$$
$$D=\sqrt{A{A}^T}$$
$$V=eigenvector(A^{T}A)$$

PCO(Principal Coordinate Analysis)

$$S=X^{T}HX$$
power equal : HH=H
$$B=HX{X}^{T}H$$
variance matrix
AB=BA
Non-zero eigenvector are equal.