浅谈SVD分解和CUR分解

1.Power iteration

In mathematics, the power iteration (also known as power method) is an eigenvalue algorithm: given a matrix $A$, the algorithm will produce a number $\lambda$ , which is the greatest (in absolute value) eigenvalue of $A$, and a nonzero vector $v$, the corresponding eigenvector of $\lambda$ , such that$Av=\lambda v$. The algorithm is also known as the Von Mises iteration.

The power iteration is a very simple algorithm, but it may converge slowly. It does not compute a matrix decomposition, and hence it can be used when {\displaystyle A} A is a very large sparse matrix.

When M is a stochastic matrix(随机矩阵), the limiting vector is the principal eigenvector (主特征向量, the eigenvector with the largest eigenvalue), and its corresponding eigenvalue is 1. This method for finding the principal eigenvector, called power iteration, works quite generally, although if the principal eigenvalue (eigenvalue associated with the principal eigenvector) is not 1, then as $i$ grows, the ratio of ${{\rm{M}}^{i + 1}}v$ to ${{\rm{M}}^{i }}v$ approaches the principal eigenvalue while M iv approaches a vector (probably not a unit vector) with the same direction as the principal eigenvector.

1.2Power iteration算法

1.3实例

$$M = \left[ {\begin{array}{*{20}{c}} 3&2\\ 2&6 \end{array}} \right],{\lambda _1} = 7,{v_1} = \left[ {\begin{array}{*{20}{c}} {0.447}\\ {0.894} \end{array}} \right],{\lambda _2} = 2,{v_2} = \left[ {\begin{array}{*{20}{c}} {0.894}\\ { - 0.447} \end{array}} \right]$$

M=[3 2; 2 6];
x0=[1 1]';
err=1;
%对M采用power iteration
while (err>0.001)
    x1=M*x0/norm(M*x0);
    err=norm([x1-x0])
    x0=x1;
end
x1    %主特征向量
lamda1=x1'*M*x1    %主特征值

M1=M-lamda1*x1*x1'
%对M1采用power iteration 
x0=[1 1]';
err=1;
while (err>0.001)
    x2=M1*x0/norm(M1*x0);
    err=norm([x2-x0])
    x0=x2;
end
x2%第二特征向量
lamda2=x2'*M1*x2%第二特征值

容易知道采用迭代法球出来的特征对与定义法求出来的特征对非常接近。

2.Principal-Component Analysis

Principal-component analysis, or PCA, is a technique for taking a dataset consisting of a set of tuples representing points in a high-dimensional space and finding the directions along which the tuples line up best. The idea is to treat the set of tuples as a matrix $M$ and find the eigenvectors for ${\rm{M}}{M^T}$ or ${\rm{M^T}}{M}$.The matrix of these eigenvectors can be thought of as a rigid rotation in a high dimensional space. When you apply this transformation to the original data, the axis corresponding to the principal eigenvector is the one along which the points are most “spread out,” More precisely, this axis is the one along which the variance of the data is maximized. Put another way, the points can best be viewed as lying along this axis, with small deviations from this axis. Likewise,the axis corresponding to the second eigenvector (the eigenvector corresponding to the second-largest eigenvalue) is the axis along which the variance of distances from the first axis is greatest, and so on.

We can view PCA as a data-mining technique. The high-dimensional data can be replaced by its projection onto the most important axes. These axes are the onescorresponding to the largest eigenvalues. Thus, the original data is approximated by data with many fewer dimensions, which summarizes well the original data.

主成分分析（Principal Component Analysis，PCA）， 是一种统计方法。通过正交变换将一组可能存在相关性的变量转换为一组线性不相关的变量，转换后的这组变量叫主成分。主成分分析首先是由K.皮尔森（Karl Pearson）对非随机变量引入的，尔后H.霍特林将此方法推广到随机向量的情形。信息的大小通常用离差平方和或方差来衡量。

3.${\rm{M}}{M^T}$ 和 ${\rm{M^T}}{M}$的共同特征值

对于${M_{n \times m}}$矩阵，矩阵$MM^T$的特征值是矩阵$M^TM$的特征值加上$n-m$个0，如果$n>m$.反之也是成立，如果$n<m$，则矩阵$M^TM$的特征值是矩阵$MM^T$的特征值加上$n-m$个0

证明如下：
对于矩阵$M_{n*m}$，我们假设 n