机器学习（二）

一、模型
线性回归

in supervised learning,have a data set(called a training set)-> to predict the price of the house

symbols of supervised learning:
m ->训练样本的数量
x ->输入变量（feature）
y->目标变量


二、代价函数

what we want to do is come up with values for parameter
(minimization problem)->minimize the square difference
(h(x)-y) ->sum over my training set,sum from i equals 1 to M of the square difference between the prediction

in actually,cost function:


depend on your training set,get a cost function looks like that:

使用等高线：

三、梯度下降

• start with some parameters
• keep changing to reduce cost function until end up at a minimum
  可能由于初始点的不同，导致最后结果的差异：
  
  
  learning rate：how big a step we take downhill with gradient descent
• if too small,gradient descent can be slow
• if too large,gradient descent can overshot the minimum,it may fail to converge,or even diverge
• as we approach a local minimum,gradient descent will automatically take small steps,so,no need to decrease learning rate over time

需要先更新temp后，再赋值，否则temp1是按照更新后的值进行计算所得到的。

simplified situation：


“Batch” Gradient Descent:each step of gradient descent uses all the training examples.

some other of gradient descent that are not batch versions,look at small subsets of the training sets at a time.

四、多元特征变量

多个特征量表达式可以写为;



五、多元梯度下降（特征缩放）
idea：make sure features are on a similar scale

it turns out gradient descent will just have a much harder time(反复振荡）

特征值除以最大值：

get every feature into approximately (-1,1)

归一化：

即（x-average value）/（range value）

六、多元梯度下降法（学习率）

• debugging：how to make sure gradient descent is working correctly(cost function decrease after every iteration)
  
• how to choose learning rate(for sufficiently small learning rate,cost function decrease on every iteration)

除了线性回归，还可以使用多项式回归等其他拟合方法