Machine Learning Notebook - Week 1(2) - Andrew Ng

Regression

Training set : m training examples ( x(i), y(i) )
features : x
target : y
hypothesis ： h: x->y
eg: Univariate linear regression

supervised learning problem process

• Cost function
  objective function which we want to minimize
  eg: squared error function

   

  

  squared error function

Visualize function J -> Contour Plot

Contour Plot of cost function

Our task is to teach the alogorithm(machine) to automatically find the value of parameters(theta) which minimize the cost function.

Gradient Descent

initialization of a position on the mountain -> walk down hill step by step -> until valley

 

Gradient Descent

Remark :
Assignment a:=b, a:=a+1
Truth assertion a=b

Gradient Descent Math form

Remark :
simultaneously update the parameters
learning rate(alpha) too small -> slowly converge
learning rate(alpha) too large -> overshoot the minimun -> diverge

Influence of choice of learning rate(alpha)

Remark :
Step gets smaller automatically -> no need to decrease learning rate at each time

"Batch" GD: each step of GD uses all the training examples

Linear Algebra

• Matrix multiplication property:
  non-community : AB != BA
  associative : (AB)C=A(BC)
• Matrix inverse
• Matrix transpose