逻辑回归与梯度下降
已知方程
假设函数:
hθ(x(i))=g(θTx)=11+e−θTx
Cost函数:
Cost(hθ(x),y)=ylog(hθ(x))+(1−y)log(1−hθ(x))
代价函数:
J(θ)=−1m∑i=1mCost(hθ(x(i)),y(i))=−1m∑i=1m[y(i)log(hθ(x(i)))+(1−y(i))log(1−hθ(x(i)))]
梯度下降公式:
θj:=θj−α∂J(θ)∂θj
梯度下降推导
∂J(θ)∂θj∂Cost(hθ(x(i)),y(i))∂θj∂hθ(x(i))∂θj∂Cost(hθ(x(i)),y(i))∂θj∂J(θ)∂θj=−1m∑i=1mCost(hθ(x(i)),y(i))=[y(i)1hθ(x(i))−(1−y(i))11−hθ(x(i))]∂hθ(x(i))∂θj=∂11+e−θTx(i)∂θj=−1[1+e−θTx(i)]2⋅∂[1+e−θTx(i)]∂θj=−1[1+e−θTx(i)]2⋅(−e−θTx(i)⋅∂θTx(i)∂θj)=1[1+e−θTx(i)]2⋅e−θTx(i)⋅x(i)j=11+e−θTx(i)⋅(1−11+e−θTx(i))⋅x(i)j=hθ(x(i))⋅(1−hθ(x(i)))⋅x(i)j=[y(i)1hθ(x(i))−(1−y(i))11−hθ(x(i))]∂hθ(x(i))∂θj=[y(i)1hθ(x(i))−(1−y(i))11−hθ(x(i))]hθ(x(i))⋅(1−hθ(x(i)))⋅x(i)j=[y(i)(1−hθ(x(i)))−(1−y(i))hθ(x(i))]⋅x(i)j=−[hθ(x(i))−y(i)]⋅x(i)j=1m∑i=1m[hθ(x(i))−y(i)]⋅x(i)j