18.065 LEC3. Orthonormal Columns In Q Give Q’Q= I

《Matrix Methods in Data Analysis, Signal Processing, and Machine Learning》MIT Course Number：18.065课程习题答案

LEC3. Orthonormal Columns In Q Give Q’Q= I

Problems of Lecture 3 (from textbook Section I.5)

2 Draw unit vectors u and v that are not orthogonal. Show that $\omega =\text{v}-\text{u}({\text{u}}^T\text{v})$ is orthogonal to u (and add w to your picture).

4 Key property of every orthogonal matrix : $||Qx|{|}^2=||x|{|}^2$ for every vector x.More than this, show that $(Qx{)}^T(Qy)=x^{T}y$ for every vector x and y. So lengths and angles are not changed by Q. Computations with Q never overflow !

6 A permutation matrix has the same columns as the identity matrix (in some order). Explain why this permutation matrix and every permutation matrix is orthogonal :$$P=\left[\begin{array}{cccc}0& 1& 0& 0\\ 0& 0& 1& 0\\ 0& 0& 0& 1\\ 1& 0& 0& 0\end{array}\right]hasorthonormalcolumnsso{P}^{T}P=\underline{\ast \ast \ast }and{P}^{-1}=\underline{\ast \ast \ast }.$$
When a matrix is symmetric or orthogonal, it will have orthogonal eigenvectors.
This is the most important source of orthogonal vectors in applied mathematics.

Solutions to Lecture 3

2 v is separated into a piece $u(u^{T}v)$ in the direction of u and the remaining piece $\omega =\text{v}-\text{u}({\text{u}}^T\text{v})$ perpendicular to u. Check $$u^{T}w=u^{T}v-(u^{T}u)(u^{T}v)=0$$

4 Check $(Qx{)}^T(Qy)=x^T{Q}^{T}Qy=x^{T}y$ : Angles are preserved when all vectors are multiplied by Q. Remember $x^{T}y=||x||||y||cos\theta =(Qx{)}^T(Qy)$: same θ!

6 Every permutation matrix has unit vectors in its columns (single 1 and (n − 1) zeros). Those columns are orthogonal because their 1’s are in different positions.$$P^{T}P=\underline{I}and{P}^{-1}=\underline{P^T}$$