统计学习方法第十六章——主成分分析

式子(16.3)和(16.4)的推导

由式子(16.2)和$\Sigma =\mathrm{cov}(\mathbf{x},\mathbf{x})=E\left[(\mathbf{x}-\mathbf{\mu })(\mathbf{x}-\mathbf{\mu }{)}^{\mathrm{T}}\right]$得
$$\begin{array}{rl}\mathrm{var}\left(y_i\right)& =E\left[\left(y_i-E\left(y_i\right)\right){\left(y_i-E\left(y_i\right)\right)}^T\right]\\ & =E\left[\left({\alpha }_i^{T}x-{\alpha }_i^T\mu \right){\left({\alpha }_i^{T}x-{\alpha }_i^T\mu \right)}^T\right]\\ & =E\left[\left({\alpha }_i^T(x-\mu )\right){\left({\alpha }_i^T(x-\mu )\right)}^T\right]\\ & =E\left[{\alpha }_i^T(x-\mu )(x-\mu {)}^T{\alpha }_i\right]\\ & ={\alpha }_i^{T}E\left[(x-\mu )(x-\mu {)}^T\right]{\alpha }_i\\ & ={\alpha }_i^T\Sigma {\alpha }_i\end{array}$$
式子(16.3)得证，同理可以证明式子(16.4)
$$\begin{array}{rl}\mathrm{cov}(y_i,y_j)& =E\left[\left(y_i-E\left(y_i\right)\right){\left(y_j-E\left(y_j\right)\right)}^T\right]\\ & =E\left[\left({\alpha }_i^{T}x-{\alpha }_i^T\mu \right){\left({\alpha }_j^{T}x-{\alpha }_j^T\mu \right)}^T\right]\\ & =E\left[\left({\alpha }_i^T(x-\mu )\right){\left({\alpha }_j^T(x-\mu )\right)}^T\right]\\ & =E\left[{\alpha }_i^T(x-\mu )(x-\mu {)}^T{\alpha }_j\right]\\ & ={\alpha }_i^{T}E\left[(x-\mu )(x-\mu {)}^T\right]{\alpha }_j\\ & ={\alpha }_i^T\Sigma {\alpha }_j\end{array}$$
式子(16.7)最优化问题推导补充

首先要知道矩阵求导的几个简单法则：
$$\begin{gathered} \frac{\partial \left({\mathbf{x}}^T\mathbf{a}\right)}{\partial \mathbf{x}}=\frac{\partial \left({\mathbf{a}}^T\mathbf{x}\right)}{\partial \mathbf{x}}=\mathbf{a} \\ \frac{\partial \left({\mathbf{x}}^{\mathbf{T}}\mathbf{x}\right)}{\partial \mathbf{x}}=2\mathbf{x} \\ \frac{\partial \left({\mathbf{x}}^T\mathbf{A}\mathbf{x}\right)}{\partial \mathbf{x}}=\mathbf{A}\mathbf{x}+{\mathbf{A}}^T\mathbf{x} \end{gathered}$$
所以拉格朗日函数求导并令其得0为：
$$\begin{array}{rl}原式& =\frac{\partial {\alpha }_1^T\Sigma {\alpha }_1}{\partial {\alpha }_1}-\frac{\lambda \partial {\alpha }_1\alpha }{\partial {\alpha }_1}\\ & =\Sigma {\alpha }_1+{\Sigma }^T{\alpha }_1-2\lambda {\alpha }_1\\ & =2\Sigma {\alpha }_1-2\lambda {\alpha }_1\\ & =0\end{array}$$

【例16.1】一些计算的补充

首先求相关矩阵R的特征值和特征向量，这边就直接使用python来求解，毕竟所给的数据不太好手动求

import numpy as np

mat = np.array([[1, 0.44, 0.29, 0.33],
                [0.44, 1, 0.35, 0.32],
                [0.29, 0.35, 1, 0.60],
                [0.33, 0.32, 0.60, 1]])

eigenvalue, featurevector = np.linalg.eig(mat)

print("特征值：", np.around(eigenvalue, decimals=2))
print("特征向量：", np.around(featurevector, decimals=3))

可以看到，特征值和书上一样的，但是特征向量不一样，不过这没影响，毕竟特征向量本来就不唯一。

$y_1的方差贡献率为\frac{{\lambda }_1}{{\sum }_{i=1}{\lambda }_i}=\frac{2.17}{4}=0.543$，同理$y_2的方差贡献率为\frac{{\lambda }_2}{{\sum }_{i=2}{\lambda }_i}=\frac{0.87}{4}=0.218$

因子负荷量的计算公式为：$\rho (y_k,x_i)=\sqrt{{\lambda }_k}{e}_{ik}$