Machine Learning Lecture Note 003 -- Normal Equation

A different way to solve for $\theta$ in cost function.

The equation for normal equation is

$\theta =(X^{T}X{)}^{-1}{X}^{T}y$

where is X is the matrix with all features values, and y is the output vector.

Gradient descent vs. Normal equation

Gradient Descent	Normal Equation
Human choose $\alpha$ (learning rate)	Do not need to choose $\alpha$
Need iteration	Do not need iteration
$O(n^2)$	$O(n^3)$ – Due to inverse
Good for lots of features	Good for fewer features