An Introduction to Linear Regression
Intent
supervised learning:
- regression (continious)
- classfication (discrete)
h(θ)=∑i=0nθiXi=θTX
For historical reasons, this function h is called a hypothesis.
θ′sis the parameters
(x(i),y(i))(trainingexample)
{ (x(i),y(i));i=1,...,m}(trainingset)
要让 h(θ) 接近于trainset中的 y(i)
cost function: J(θ)=12∑i=1m(hθ(x(i)−y(i))2
为了让train出的model能stable,需要在bias和variance中tradeoff。regularized regression 会使用L1(lasso)和L2(ridge)作为penalty。其中L1相当于feature selection(parameter变为0)。而L2主要是减少variance。cross validation和bootstrap也是常用的trade bias和variance的方法。
LMS
gradient descent
θj=θj−α∂∂θjJ(θ)
α is learning rate
∂∂θj