Linear Algebra 2: Gram Schmidt Process

Summarized from CUHK(SZ) MAT2041(2022Fall) Lecture 17.

Motivation

Any $n$ linear independent $n$-vector can form a basis of a $n$-dim space. However, people are accustomed to orthogonal bases. In high school students learn only Cartesian coordinate system. If we can transform a random basis into an orthogonal (even orthonormal) basis, it’s more convenient in some situations.

Gram Schmidt Process: to change a random basis into an orthonormal basis.

Solution

Assume that the ordinary basis is { u 1 , u 2 , … , u n } \{u_1,u_2,\ldots,u_n\} { u1​,u2​,…,un​}, we want { v 1 , v 2 , … , v n } ( ∀ i ≠ j ,   v i T v j = 0 ; ∀ i , ∣ ∣ v i ∣ ∣ = 1 ) \{v_1,v_2,\ldots,v_n\}(\forall i\neq j,\ v_i^Tv_j=0;\forall i, ||v_i||=1) { v1​,v2​,…,vn​}(∀i=j, viT​vj​=0;∀i,∣∣vi​∣∣=1).

There’s no limitation to the first vector, for convenience, we just pick $v_1=\frac{u_1}{||u_1||}$.

Then we try to modify $u_2$ to meet the requirements.

$u_2$ should be orthogonal to $v_1$, and $v_2$ could be expressed by only $v_1$ and $u_2$.

Thus we assume $(u_2-\alpha v_1)\perp v_1$, then v 1 T u 2 = α v 1 T v 1 = α v_1^Tu_2=\alpha v_1^Tv_1=\alpha