Data Mining and Machine Learning笔记 1

Machine Learning 分类

1 PCA

Consider linear combinations:
$$\begin{gathered} Y_1=a_{11}{X}_1+a_{12}{X}_2+...+a_{1p}{X}_p={{\mathrm{a}}_1}^{\mathsf{T}}X \\ {Y}_2=a_{21}{X}_1+a_{22}{X}_2+...+a_{2p}{X}_p={{\mathrm{a}}_2}^{\mathsf{T}}X \\ . \\ . \\ . \\ {Y}_3=a_{p1}{X}_1+a_{p2}{X}_2+...+a_{pp}{X}_p={{\mathrm{a}}_{\mathrm{p}}}^{\mathsf{T}}X \end{gathered}$$
PCA

• The linear combinations $Y_1,Y_2,...,Y_p$ are principal components
• ${\mathrm{a}}_{\mathrm{j}}$ is the eigenvector of the $j^{th}$ principal components
• $a_{j1},...,a_{jp}$ are the loadings of the $j_{th}$ principal component. Loadings make up the principal component loading vector:$a_j=(a_{j1},...,a_{jp}{)}^{\mathsf{T}}$
• socre
  $$y_{ij}=a_{j1}{x}_{i1}+a_{j2}{x}_{i2}+...+a_{jp}{x}_{ip}$$
  calculate the observation in new coordinate system of principal components.

Properties

• $Y_1,Y_2,...,Y_p$ are pairwise-uncorrelated, $Var(Y)=diag({\lambda }_1,...,{\lambda }_p)=\Lambda$， where ${\lambda }_j$ is the $j^{th}$ eigenvalue of $\Sigma$, $Var(Y_1)=a^{\mathsf{T}}\Sigma a$
• The total variance is preserved under the principal component transformation, ${\Sigma }_{j=1}^{p}Var(Y_j)={\Sigma }_{j=1}^{p}Var(X_j)$
• The first $k$ principal components account for the proportion $\frac{{\Sigma }_{j=1}^k{\lambda }_j}{{\Sigma }_{j=1}^k{\lambda }_p}$ of the total variance

PCA practice

• Proportion of Variation
• Cattell’s method
• Kaiser’s method

steps see https://blog.csdn.net/MINGRAN_JIA/article/details/123242755

PCA biplot