Chapter 3 Orthogonality

Chapter 3 Orthogonality

3.1 Orthogonal Vectors and Subspaces

the length $||x||$ in $R^n$ is the positive square root of $x^{T}x$:
$$\text{Length square }||x|{|}^2=x_1^2+x_2^2+\cdots +x_n^2=x^{T}x\text{\hspace{1em}(1)}$$

Orthogonal Vectors

$$\text{Sides of a right triangule}||x|{|}^2+||y|{|}^2=||x-y|{|}^2\text{\hspace{1em}(2)}$$

$$\text{Orthogonal vectors}{x}^{T}y=x_1{y}_1+\cdots +x_n{y}_n=0\text{\hspace{1em}(3)}$$

$$\text{Inner product }{x}^{T}y=\left[\begin{array}{ccc}{x}_1& \dots & x_n\end{array}\right]\left[\begin{array}{c}{y}_1\\ ⋮\\ y_n\end{array}\right]=x_1{y}_1+\cdots +x_n{y}_n\text{\hspace{1em}(4)}$$

3A The inner product $x^Ty $ is zero if and only if $x$ and $y$ are orthogonal vectors, If $x^{T}y>0$, their angle is less than $90^{\circ }$, If $x^{T}y<0$, their angle is greater than $90^{\circ }$.

If nonzero vector $v_1,\dots v_k$ are mutually orthogonal (every vector is perpendicular to every other), then those vectors are linearly independent.

Orthogonal Subspaces

3B Two subspaces $V$ and $W$ of the same space $R^n$ are orthogonal if every vector $v$ in $V$ is orthogonal to every vector $w$ in $W$: $v^{T}w=0$ for all $v$ and $w$.

3C Fundamental theorem of orthogonality The row space is orthogonal to the nullspace (in $R^n$). The column space is orthogonal to the left nullspace (in $R^m$).

**Definition. Given a subspace $V$ of $R^n$, the space of all vectors orthogonal to $V$ is called the orthogonal complement of $V$. } It is denoted by $V^{\perp }="Vperp"$ **

3D Fundamental Theorem of Linear Algebra, Part II

​ The nullspace is the orthogonal complement of the row space in $R^n$.

​ The left nullspace if the orthogonal complement of the column space in $R^m$.

3E $Ax=b$ is solvable if and only if $y^{T}b=0$ whenever $y^{T}A=0$

The Matrix and the Subspaces

If $W=V^{\perp }$ then $V=W^{\perp }$ and $dimV+dimW=n$.

In other words $V^{\perp \perp }=V$. The dimensions of $V$ and $W$ are right, and the whole space $R^n$ is being decomposed into two perpendicular parts.

When multiplied by $A$, this is $Ax=A{x}_r+A{x}_n$, :

​ $x_n$ is the nullspace component, $A{x}_n=0$

​ $x_r$ is the row space component, $A{x}_r=Ax$.

3F From the row space to column space, $A$ is actually invertible. Every vector $b$ in the column space comes from exactly one vector $x_r$ in the row space.

Every matrix transforms its row space onto its column space.

3.2 Cosines and Projections onto lines

The line connecting $b$ to $p$ is perpendicular to $a$.

The point $p$ is the projection of $b$ onto the subspace.

inner products and cosines