normal equation

normal equation:
Method to solve for $\theta$ analytically.
one step to get the optimal value.

$$\theta=(X^TX)^{-1}X^Ty$$
Octave:

pinv(X'*X)*X'*y

Gradient Descent
- Need to choose $\alpha$
- Needs many iterations
- Works well even when n is large
Normal Equation

• No need to choose $alpha$
• Don’t need to iteration
• Need to compute $(X^TX)^{-1}$

• Slow if n is very large(beyond $10^4$)

  What if $X^TX$ is non-invertible?

• Redundant features
  (e.g. x1=size in feet
  x2= size in m)

• Too many features(e.g.m<=n)
  delete some features, or use regularization.