一、模型
线性回归
in supervised learning,have a data set(called a training set)-> to predict the price of the house
symbols of supervised learning:
m ->训练样本的数量
x ->输入变量(feature)
y->目标变量
二、代价函数
what we want to do is come up with values for parameter
(minimization problem)->minimize the square difference
(h(x)-y) ->sum over my training set,sum from i equals 1 to M of the square difference between the prediction
in actually,cost function:
depend on your training set,get a cost function looks like that:
使用等高线:
三、梯度下降
- start with some parameters
- keep changing to reduce cost function until end up at a minimum
可能由于初始点的不同,导致最后结果的差异:
learning rate:how big a step we take downhill with gradient descent - if too small,gradient descent can be slow
- if too large,gradient descent can overshot the minimum,it may fail to converge,or even diverge
- as we approach a local minimum,gradient descent will automatically take small steps,so,no need to decrease learning rate over time
需要先更新temp后,再赋值,否则temp1是按照更新后的值进行计算所得到的。
simplified situation:
“Batch” Gradient Descent:each step of gradient descent uses all the training examples.
some other of gradient descent that are not batch versions,look at small subsets of the training sets at a time.
四、多元特征变量
多个特征量表达式可以写为;
五、多元梯度下降(特征缩放)
idea:make sure features are on a similar scale
it turns out gradient descent will just have a much harder time(反复振荡)
特征值除以最大值:
get every feature into approximately (-1,1)
归一化:
即(x-average value)/(range value)
六、多元梯度下降法(学习率)
- debugging:how to make sure gradient descent is working correctly(cost function decrease after every iteration)
- how to choose learning rate(for sufficiently small learning rate,cost function decrease on every iteration)
除了线性回归,还可以使用多项式回归等其他拟合方法