Multiple features
for convenience of notation, define
, the new hypothesis is
Gradient descent for multiple variables
new algorithm is
Gradient descent in practive I: Feature Scaling
make sure features are on a similar scale
- feature scaling
- mean normalization
Gradient descent in practice II: Learning rate
- make sure gradient descent is working correctly
- gradient descent not working - use smaller \alpha
- to choose
, try ..., 0.001, 0.003, 0.01, 0.03, ..., 0.1, ..., 1, ...
Features and polynomial regression
choice of features
Normal equations
method for solve analytically
if there are m examples, each example have n input features, so is (n+1)-by-1 matrix, X is m-by-(n+1) matrix, y is m-dimensional vector
?
Advantages and disadvantages of gradient descent and normal equation
training examples,
features
Gradient descent
- need to choose
- need many iterations
- works well even when
is large
Normal equation
- no need to choose
- don't need to iterate
- need to compute
- slow if
is very large
- if
is larger than 10000 or 100000, need consider to choose gradient descent
- only suitable for linear regression
Normal equation and non-invertibility (optional)
- redundant features (linearly dependent)
- too many features (e.g.
)
- delete some features, or use regularization