吴恩达机器学习笔记之逻辑回归

逻辑回归

原来的线性回归函数:
$$h_{\theta }(x)={\theta }^T\ast x$$

${\theta }^T\ast x==\theta \cdot x$
表示两个向量的内积, 即两个向量做Dot-Product

在此基础上增加Sigmoid函数,改成如下的逻辑回归函数:
$$h_{\theta }(x)=\frac{1}{(1+e^{-{\theta }^T\ast x})}$$
其中:
$h_{\theta }(x)=g({\theta }^T\ast x)$
$g(z)=\frac{1}{(1+e^{-z})}$

决策界限

传统的线性回归函数 ${\theta }^{T}x$ 假如表示成如下:

${\theta }_0+{\theta }_1{x}_1+{\theta }_2{x}_2$

令$\theta =\left[\begin{array}{c}-3\\ 1\\ 1\end{array}\right]$,可知：
$x_1+x_2=3$就是这个决策界限函数.

逻辑回归代价函数的简单写法

$Cost(h_{\theta }(x),y)=-y\ast log(h_{\theta }(x))-(1-y)\ast log(1-h_{\theta }(x))$

注意:这里的log()函数相当于ln(),即以e为底的对数.

最终代价函数如下:
$$J(\theta )=1/m\ast \sum _{i=1}^{m}Cost(h_{\theta }(x^{(i)}),y^{(i)})$$
$$J(\theta )=-1/m\ast \sum _{i=1}^m[y^{(i)}log(h_{\theta }(x^{(i)}))+(1-y^{(i)})log(1-h_{\theta }(x^{(i)})]$$

最终对上式求得偏导数如下:
$$\frac{\partial }{\partial {\theta }_j}J(\theta )=1/m\ast \sum _{i=1}^m(h_{\theta }(x^{(i)})-y^{(i)})x_j^{(i)}$$
其中求导过程如下:
这里对复合函数求导：
令:
$f_1({\theta }_j)=y^{(i)}log(h_{\theta }(x^{(i)}))$
$u({\theta }_j)=h_{\theta }(x^{(i)});g(u)=y^{(i)}log(u);$
$\therefore f_1^{\prime }({\theta }_j)=g^{\prime }(u)\ast u^{\prime }({\theta }_j)$
$\therefore f_1^{\prime }({\theta }_j)=\frac{y^{(i)}}{u({\theta }_j)}\ast u^{\prime }({\theta }_j)=\frac{y^{(i)}}{h_{\theta }(x^{(i)})}\ast u^{\prime }({\theta }_j)$

因为:$lo{g}^{\prime }(x)=1/x$
$(e^x{)}^{\prime }=e^x$
$g^{\prime }(z)=g(z)\ast (1-g(z))$

$u^{\prime }({\theta }_j)=\frac{e^{-{\theta }^T\ast x}}{(1+e^{-{\theta }^T\ast x}{)}^2}\ast x_j^{(i)}=h_{\theta }(x^{(i)})\ast (1-h_{\theta }(x^{(i)}))\ast x_j^{(i)}$
$\therefore f_1^{\prime }({\theta }_j)=y^{(i)}\ast (1-h_{\theta }(x^{(i)}))\ast x_j^{(i)}$

又有:
$f_2({\theta }_j)=(1-y^{(i)})log(1-h_{\theta }(x^{(i)})$
最终:
$f_2^{\prime }({\theta }_j)=(1-y^{(i)})\ast \frac{-h_{\theta }(x^{(i)})\ast (1-h_{\theta }(x^{(i)}))\ast x_j^{(i)}}{1-h_{\theta }(x^{(i)})}$
消去分子分母后得到:
$f_2^{\prime }({\theta }_j)=-(1-y^{(i)})\ast h_{\theta }(x^{(i)})\ast x_j^{(i)}$

所以:
$$\frac{\partial }{\partial {\theta }_j}J(\theta )=1/m\ast \sum _{i=1}^m(f_1^{\prime }({\theta }_j)+f_2^{\prime }({\theta }_j))$$
$$\frac{\partial }{\partial {\theta }_j}J(\theta )=-1/m\ast \sum _{i=1}^m(y^{(i)}-h_{\theta }(x^{(i)}))x_j^{(i)}$$
然后提出一个负号,最终求得偏导数如下:
$$\frac{\partial }{\partial {\theta }_j}J(\theta )=1/m\ast \sum _{i=1}^m(h_{\theta }(x^{(i)})-y^{(i)})x_j^{(i)}$$