逻辑回归与梯度下降

最新推荐文章于 2021-11-22 12:25:20 发布

Vicent_Chen

最新推荐文章于 2021-11-22 12:25:20 发布

阅读量320

点赞数

分类专栏：机器学习文章标签：机器学习

本文链接：https://blog.csdn.net/cjw_soledad/article/details/77972495

版权

机器学习专栏收录该内容

2 篇文章 0 订阅

订阅专栏

逻辑回归与梯度下降

已知方程

假设函数：

h θ (x (i)) = g (θ T x) = 1 1 + e - θ T x

$h_\theta(x^{(i)})=g(\theta^Tx)=\frac{1}{1+e^{-\theta^Tx}}$
Cost函数：

C o s t (h θ (x), y) = y l o g (h θ (x)) + (1 - y) l o g (1 - h θ (x))

$Cost(h_\theta(x),y)=ylog(h_\theta(x))+(1-y)log(1-h_\theta(x))$
代价函数：

J (θ) = - 1 m \sum i = 1 m C o s t (h θ (x (i)), y (i)) = - 1 m \sum i = 1 m [y (i) l o g (h θ (x (i))) + (1 - y (i)) l o g (1 - h θ (x (i)))]

$\begin{align} J(\theta)&=-\frac{1}{m}\sum_{i=1}^m Cost(h_\theta(x^{(i)}),y^{(i)})\\ &=-\frac{1}{m}\sum_{i=1}^m[y^{(i)}log(h_\theta(x^{(i)}))+(1-y^{(i)})log(1-h_\theta(x^{(i)}))] \end{align}$
梯度下降公式：

θ j : = θ j - α \partial J ( θ ) \partial θ j

$\theta_j:=\theta_j-\alpha\frac{\partial J(\theta)}{\partial \theta_j}$

梯度下降推导

\partial J ( θ ) \partial θ j \partial C o s t ( h θ ( x ( i ) ) , y ( i ) ) \partial θ j \partial h θ ( x ( i ) ) \partial θ j \partial C o s t ( h θ ( x ( i ) ) , y ( i ) ) \partial θ j \partial J ( θ ) \partial θ j = - 1 m \sum i = 1 m C o s t (h θ (x (i)), y (i)) = [y (i) 1 h θ ( x ( i ) ) - (1 - y (i)) 1 1 - h θ ( x ( i ) )] \partial h θ ( x ( i ) ) \partial θ j = \partial 1 1 + e - θ T x ( i ) \partial θ j = - 1 [ 1 + e - θ T x ( i ) ] 2 \cdot \partial [ 1 + e - θ T x ( i ) ] \partial θ j = - 1 [ 1 + e - θ T x ( i ) ] 2 \cdot (- e - θ T x (i) \cdot \partial θ T x ( i ) \partial θ j) = 1 [ 1 + e - θ T x ( i ) ] 2 \cdot e - θ T x (i) \cdot x (i) j = 1 1 + e - θ T x ( i ) \cdot (1 - 1 1 + e - θ T x ( i )) \cdot x (i) j = h θ (x (i)) \cdot (1 - h θ (x (i))) \cdot x (i) j = [y (i) 1 h θ ( x ( i ) ) - (1 - y (i)) 1 1 - h θ ( x ( i ) )] \partial h θ ( x ( i ) ) \partial θ j = [y (i) 1 h θ ( x ( i ) ) - (1 - y (i)) 1 1 - h θ ( x ( i ) )] h θ (x (i)) \cdot (1 - h θ (x (i))) \cdot x (i) j = [y (i) (1 - h θ (x (i))) - (1 - y (i)) h θ (x (i))] \cdot x (i) j = - [h θ (x (i)) - y (i)] \cdot x (i) j = 1 m \sum i = 1 m [h θ (x (i)) - y (i)] \cdot x (i) j

$\begin{align} \frac{\partial J(\theta)}{\partial \theta_j}&=-\frac{1}{m}\sum_{i=1}^m Cost(h_\theta(x^{(i)}),y^{(i)}) \\ \frac{\partial Cost(h_\theta(x^{(i)}),y^{(i)})}{\partial \theta_j}&=[y^{(i)}\frac{1}{h_\theta(x^{(i)})}-(1-y^{(i)})\frac{1}{1-h_\theta(x^{}(i))}]\frac{\partial h_\theta(x^{(i)})}{\partial \theta_j} \\ \frac{\partial h_\theta(x^{(i)})}{\partial \theta_j}&=\frac{\partial \frac{1}{1+e^{-\theta^Tx^{(i)}}}}{\partial \theta_j} \\ &=-\frac{1}{[1+e^{-\theta^Tx^{(i)}}]^2}\cdot\frac{\partial[1+e^{-\theta^Tx^{(i)}}]}{\partial \theta_j} \\ &=-\frac{1}{[1+e^{-\theta^Tx^{(i)}}]^2}\cdot(-e^{-\theta^Tx^{(i)}} \cdot\frac{\partial\theta^Tx^{(i)}}{\partial \theta_j}) \\ &=\frac{1}{[1+e^{-\theta^Tx^{(i)}}]^2}\cdot e^{-\theta^Tx^{(i)}}\cdot x^{(i)}_j \\ &=\frac{1}{1+e^{-\theta^Tx^{(i)}}}\cdot (1-\frac{1}{1+e^{-\theta^Tx^{(i)}}})\cdot x^{(i)}_j \\ &= h_\theta(x^{(i)})\cdot (1-h_\theta(x^{(i)}))\cdot x^{(i)}_j\\ \frac{\partial Cost(h_\theta(x^{(i)}),y^{(i)})}{\partial \theta_j}&=[y^{(i)}\frac{1}{h_\theta(x^{(i)})}-(1-y^{(i)})\frac{1}{1-h_\theta(x^{}(i))}]\frac{\partial h_\theta(x^{(i)})}{\partial \theta_j} \\ &=[y^{(i)}\frac{1}{h_\theta(x^{(i)})}-(1-y^{(i)})\frac{1}{1-h_\theta(x^{}(i))}]h_\theta(x^{(i)})\cdot (1-h_\theta(x^{(i)}))\cdot x^{(i)}_j \\ &=[y^{(i)}(1-h_\theta(x^{(i)}))-(1-y^{(i)})h_\theta(x^{(i)})]\cdot x^{(i)}_j \\ &=-[h_\theta(x^{(i)})-y^{(i)}]\cdot x_j^{(i)} \\ \\ \frac{\partial J(\theta)}{\partial \theta_j}&=\frac{1}{m}\sum_{i=1}^m [h_\theta(x^{(i)})-y^{(i)}]\cdot x_j^{(i)} \end{align}$