AI-005: 吴恩达教授（Andrew Ng）的机器学习课程学习笔记15-20

本文是学习Andrew Ng的机器学习系列教程的学习笔记。教学视频地址：

https://study.163.com/course/introduction.htm?courseId=1004570029#/courseDetail?tab=1

本文中的白色背景视频截图来自Andrew Ng的视频脚程, 思维导图为原创总结。

多变量的线性回归：

15. Linear Regression with multiple variables - multiple features
首先定义出线性回归的矩阵表示：
 

Use vector to show feature

Inner product(内积) is θ transpose X

Feature vector X

Parameter vector theta θ

Multivariate linear regression

16. Linear Regression with multiple variables - Gradient descent for multiple variables
然后是梯度下降的矩阵表示：

就是对各个特征求偏导，不包含求导参数时，导数都为0.
 

17. Linear Regression with multiple variables - Gradient descent in practice I : feature scaling
 

Feature scaling 特征缩放

Y = θ1 * x1 + θ2 * x2

If x1 = 0-2000, x2 = 1-5, then when θ1 change a little, if want get a same Y you need change θ2 a lot.

Use feature scaling to get more effect Gradient Descent.

Feature scaling make x1 and x2 be closer. Then the gradient descent will converge much faster.

Dividing by the maximum value

18. Linear regression with multiple variables - gradient descent in practive II: learning rate
用图形化方法来查看选择的学习率参数合不合适，每次可以按3倍的关系调整学习率。这里要不断尝试，没有绝对方法。

J(θ) should decease after each iterations

After 400 iterations, the gradient descent is already converge.

Choosing what this threshold is pretty difficult.

So, in order to check your gradient descent has converged, I actually tend to look at plots like this figure. And this figure will tell you if your gradient descent is working correctly.

When you see this figure, thy to use small learning rate

Choose more α  to try.

Choose a small one and then increase the learning rate threefold.

Many times later, you will get one value that is too small and one value is too large.

Then I try to pick the largest possible value or just something slightly smaller than the largest reasonable value that I found. This may be a good learning rate for my problem.

19. Linear regression with multiple variables - features and polynomial regression 

How to choose features and algorithm

Polynomial regression allows you to use the machinery of linear regression to fit very complicated, even very non-linear functions.

Two features times together or add higher ^

Not use frontage and depth, but use land area = frontage * depth.

Feature scaling will more important when use polynomial regression

Choose different features you can sometimes get better models.

Find better models to fit your data.

20. Linear regression with multiple variables - normal equation
用公式来计算参数：

 

Normal equation can get the optimal value θ in basically one step.

Calculus tell us how to find better through take the partial derivative of J.

Normal equation:

And if you use normal equation there is no need to features scaling.

When number of features is large, use Gradient Descent.

When get non-invertibility X^TX, how to use normal equation.

When features have linear relations, the X^TX will be non-invertible.