Coursera-Mathematics for Machine Learning: PCA Week3

Projection onto 1D subspaces

orthogonal projection of x onto u

THis projection matrix is denoted as pi u of x

Projection of a 2-D vector $\mathbf{x}$ onto a 1-dimensional subspace $U$ with basis vector $\mathbf{b}$:

$${\pi }_U(\mathbf{x})=\frac{\mathbf{b}{\mathbf{b}}^T}{{\Vert \mathbf{b}\Vert }^2}\mathbf{x}$$

For the general projection onto an M-dimensional subspace $U$ with basis vectors ${\mathbf{b}}_1,\dots ,{\mathbf{b}}_M$ we have

$${\pi }_U(\mathbf{x})=\mathbf{B}({\mathbf{B}}^T\mathbf{B}{)}^{-1}{\mathbf{B}}^T\mathbf{x}$$

where

$$\mathbf{B}=[{\mathbf{b}}_1,...,{\mathbf{b}}_M]$$

Your task is to implement orthogonal projections. We can split this into two steps

1. Find the projection matrix $\mathbf{P}$ that projects any $\mathbf{x}$ onto $U$.
2. The projected vector ${\pi }_U(\mathbf{x})$ of $\mathbf{x}$ can then be written as ${\pi }_U(\mathbf{x})=\mathbf{P}\mathbf{x}$.

the reconstruction error

the distance between the original data point and its projection onto a lower-dimensional subspace.