Chapter 5 (Eigenvalues and Eigenvectors): Iterative estimates for eigenvalues (特征值的迭代估计)

本文为《Linear algebra and its applications》的读书笔记

• In scientific applications of linear algebra, eigenvalues are seldom known precisely. Fortunately, a close numerical approximation is usually quite satisfactory.

The Power Method

幂方法

• In fact, some applications require only a rough approximation to the largest eigenvalue. The first algorithm described below can work well for this case. Also, it provides a foundation for a more powerful method that can give fast estimates for other eigenvalues as well.

---

• The power method applies to an $n\times n$ matrix $A$ with a strictly dominant eigenvalue ${\lambda }_1$, which means that ${\lambda }_1$ must be larger in absolute value than all the other eigenvalues. In this case, the power method produces a scalar sequence that approaches ${\lambda }_1$ and a vector sequence that approaches a corresponding eigenvector.
• Assume for simplicity that $A$ is diagonalizable and ${\mathbb{R}}^n$ has a basis of eigenvectors ${\mathbf{v}}_1,...,{\mathbf{v}}_n$, arranged so their corresponding eigenvalues ${\lambda }_1,...,{\lambda }_n$ decrease in size, with the strictly dominant eigenvalue first. That is,
  If $\mathbf{x}$ in ${\mathbb{R}}^n$ is written as $\mathbf{x}=c_1{\mathbf{v}}_1+...+c_n{\mathbf{v}}_n$, then
  Assume $c_1\ne 0$. Then,
  The fractions ${\lambda }_2/{\lambda }_1,...,{\lambda }_n/{\lambda }_1$ are all less than 1 in magnitude and so their powers go to zero. Hence
  Thus, for large $k$, a scalar multiple of $A^k\mathbf{x}$ determines almost the same direction as the eigenvector $c_1{\mathbf{v}}_1$, provided $c_1\ne 0$.

---

EXAMPLE 1

• Let $A=\left[\begin{array}{cc}1.8& .8\\ .2& 1.2\end{array}\right]$, ${\mathbf{v}}_1=\left[\begin{array}{c}4\\ 1\end{array}\right]$, and $\mathbf{x}=\left[\begin{array}{c}-.5\\ 1\end{array}\right]$. Then $A$ has eigenvalues 2 and 1, and the eigenspace for ${\lambda }_1=2$ is the line through $0$ and ${\mathbf{v}}_1$.
  
• We can scale each $A^k\mathbf{x}$ to make its largest entry a 1. It turns out that the resulting sequence { x k } \{\boldsymbol x_k\} {xk​} will converge to a multiple of ${\mathbf{v}}_1$ whose largest entry is $1$. Figure 2 shows the scaled sequence for Example 1. The eigenvalue ${\lambda }_1$ can be estimated from the sequence { x k } \{\boldsymbol x_k\} {xk​}, too.
  

---

PROOF

参考：https://wenku.baidu.com/view/ee7ecbeca98271fe910ef9fc.html?from=search

---

• In general, the rate of convergence depends on the ratio $|{\lambda }_2/{\lambda }_1|$. If $|{\lambda }_2/{\lambda }_1|$ is close to 1, then ${\mu }_k$ and ${\mathbf{x}}_k$ can converge very slowly, and other approximation methods may be preferred.
• With the power method, there is a slight chance that the chosen initial vector $\mathbf{x}$ will have no component in the ${\mathbf{v}}_1$ direction (when $c_1=0$). But computer rounding errors during the calculations of the ${\mathbf{x}}_k$ are likely to create a vector with at least a small component in the direction of ${\mathbf{v}}_1$. If that occurs, the ${\mathbf{x}}_k$ will start to converge to a multiple of ${\mathbf{v}}_1$.

The Inverse Power Method

逆幂法

• This method provides an approximation for $any$ eigenvalue, provided a good initial estimate $\alpha$ of the eigenvalue $\lambda$ is known.
• In this case, we let $B=(A-\alpha I{)}^{-1}$ and apply the power method to $B$. It can be shown that if the eigenvalues of $A$ are ${\lambda }_1,...,{\lambda }_n$, then the eigenvalues of $B$ are
  and the corresponding eigenvectors are the same as those for $A$.
• If $\alpha$ is really close to ${\lambda }_k$, then $1/({\lambda }_k-\alpha )$ is $much$ larger than the other eigenvalues of $B$, and the inverse power method produces a very rapid approximation to ${\lambda }_k$ for almost all choices of ${\mathbf{x}}_0$. The following algorithm gives the details.

• Notice that instead of computing $(A-\alpha I{)}^{-1}{\mathbf{x}}_k$ to get the next vector in the sequence, it is better to solve the equation $(A-\alpha I){\mathbf{y}}_k={\mathbf{x}}_k$ for ${\mathbf{y}}_k$. Since this equation for ${\mathbf{y}}_k$ must be solved for each $k$, an $LU$ factorization of $A-\alpha I$ will speed up the process.
• The Inverse Power Method can be used to approximate ${\lambda }_k$ with the smallest absolute value by applying the power method to $A^{-1}$. ($\alpha =0$)

QR algorithm

• A more robust and widely used iterative method is the QR algorithm. For instance, it is the heart of the MATLAB command eig(A), which rapidly computes eigenvalues and eigenvectors of $A$. A brief description of the QR algorithm was given in Section 5.2. Further details are presented in most modern numerical analysis texts.