Machine Learning -- Linear Regression with Multiple Variables(Andrew Ng)

Machine Learning – Linear Regression with Multiple Variables(Andrew Ng)

1. Feature Scaling

Idea: Make sure features are on a similar scale.

Example: x1 ~ [0,2000]; x2 ~ [1,5];

If you use these two features without scaling, you will get very skewed elliptical shapes. These tall and skinny ellipses, can form the contours of the cost function J(theta). And if you ran gradient descents on this cost function, your gradients may end up taking a long time and oscillate back and forth.

Methods:

1. x = x / max(x);
2. x = (x- mean(x)) / std(x); mean normalization

2. Difference between Gradient Descent and Normal Equation

Gradient Descent	Normal Equation
need to choose alpha	no need to choose alpha
needs many iteration	don’t need iteration
work well when n (# features) is large	need to compute inv(X’*X)
	slow if n is large

3. Code

1. Computing Cost function

J = (0.5/m) * (y-X*theta)'*(y-X*theta);

1. Gradient Descent

theta = theta + (alpha/m)*(X'*(y - X*theta));