Machine Learning – Linear Regression with Multiple Variables(Andrew Ng)
1. Feature Scaling
Idea: Make sure features are on a similar scale.
Example: x1 ~ [0,2000]; x2 ~ [1,5];
If you use these two features without scaling, you will get very skewed elliptical shapes. These tall and skinny ellipses, can form the contours of the cost function J(theta). And if you ran gradient descents on this cost function, your gradients may end up taking a long time and oscillate back and forth.
Methods:
- x = x / max(x);
- x = (x- mean(x)) / std(x); mean normalization
2. Difference between Gradient Descent and Normal Equation
Gradient Descent | Normal Equation |
---|---|
need to choose alpha | no need to choose alpha |
needs many iteration | don’t need iteration |
work well when n (# features) is large | need to compute inv(X’*X) |
slow if n is large |
3. Code
- Computing Cost function
J = (0.5/m) * (y-X*theta)'*(y-X*theta);
- Gradient Descent
theta = theta + (alpha/m)*(X'*(y - X*theta));