Regression回归
学习资源来源:https://www.bilibili.com/video/BV1Ht411g7Ef?p=3
用处:
Stock market forecast
Self-driving car自动驾驶-方向盘角度
Recommendation推荐系统
Step1:model
Y=b+w*xcp,,w,b are parameters(can be any value)
A set of function : model:f1,f2,
Linear model:y=b+Σwixi. xi: an attribute of input x (feature) wi:wight,b:bias
Step2:Goodness of finction
Training data->real data
Loss function L: input a function, output: how bad it is
L(f)=L(w,b)= Σ101(y^n-(b+w*xncp))2真正-估测
Step3:best function
让L(w,b)最小
gradient descent
1.一个参数
consider loss function L(w) with one parameter w
w*=arg minL(w)
随机选取wo,计算在w=w0,参数w对loss function的微分(切线斜率),找loss比较低的w值
更新w0,η(learning rate),重复达到local optimal(参数不再更新)
2.两个参数
In linear regression, the loss function L is convex(no local otimal等高线的)
Results
Y=b+wxcp,看testing data的误差 比对average error on training data
改进:一个更复杂的model:y=b+w1xcp+w2*(xcp)2同理上面
更更复杂 :y=b+w1xcp+w2(xcp)2 +w3*(xcp)3。。。。。。
模型越复杂,training average error 越小,但是testing不一定,会发生overfitting(过拟合)
Back to step1:redsign the model
Back to step 2:regularization正则化
Wi越接近0,function越平滑,(如果有一些干扰,一个平滑的function会受到较少的影响但是不能太平滑——调整λ,与调整b无关)