《吴恩达机器学习》笔记——7 Logistic回归

《吴恩达机器学习》笔记——7 Logistic回归

1 分类

分类	$y$
负类	0
正类	1

若$h_{\theta }(x)$	预测$y$
$⩾0.5$	1
$<0.5$	0

2 假设陈述

-	表达式
$h_{\theta }(x)$	$g({\theta }^{\mathrm{T}}x)$
$g(z)$ (Sigmoid/Logistic)	$$\frac{1}{1+e^{-z}}$$
$h_{\theta }(x)$	$$\frac{1}{1+e^{-{\theta }^{\mathrm{T}}x}}$$

$$h_{\theta }(x)=P(y=1|x;\theta )$$

3 决策界限

若$h_{\theta }(x)⩾0.5$，预测$y=1$	${\theta }^{\mathrm{T}}x⩾0$
若$h_{\theta }(x)<0.5$，预测$y=0$	${\theta }^{\mathrm{T}}x<0$
边界	${\theta }^{\mathrm{T}}x=0$

4 代价函数

-	表达式
训练集	{ ( x ( 1 ) , y ( 1 ) ) , ( x ( 2 ) , y ( 2 ) ) , ⋯   , ( x ( m ) , y ( m ) ) } \{(x^{(1)},y^{(1)}),(x^{(2)},y^{(2)}),\cdots,(x^{(m)},y^{(m)})\} {(x(1),y(1)),(x(2),y(2)),⋯,(x(m),y(m))}
$m$个样本	x ∈ [ x 0 x 1 ⋯ x n ] x 0 = 1 , y ∈ { 0 , 1 } x\in\left[\begin{matrix}x_0\\x_1\\\cdots\\x_n\end{matrix}\right]\quad x_0=1,y\in\{0,1\} x∈⎣⎢⎢⎡​x0​x1​⋯xn​​⎦⎥⎥⎤​x0​=1,y∈{0,1}
$h_{\theta }(x)$	$$\frac{1}{1+e^{-{\theta }^{\mathrm{T}}x}}$$
$J(\theta )$	$$\frac{1}{m}\sum _{i=1}^m\mathrm{Cost}(h_{\theta }(x^{(i)}),y^{(i)})$$
$\sout{\mathrm{Cost}(h_{\theta }(x),y)}$	$\sout{\frac{1}{2}(h_{\theta }(x)-y{)}^2：导致\sout{J(\theta )}为非凸函数}$
$\mathrm{Cost}(h_{\theta }(x),y)$	$\{\begin{array}{cc}-\log (h_{\theta }(x))& 若y=1\\ -\log (1-h_{\theta }(x))& 若y=0\end{array}$

5 简化代价函数与梯度下降[code]

-	表达
$J(\theta )$	$\frac{1}{m}\sum _{i=1}^m\mathrm{Cost}(h_{\theta }(x^{(i)}),y^{(i)})$
$\mathrm{Cost}(h_{\theta }(x),y)$	$\{\begin{array}{cc}-\log (h_{\theta }(x))& 若y=1\\ -\log (1-h_{\theta }(x))& 若y=0\end{array}$
$\mathrm{Cost}(h_{\theta }(x),y)$	$-y\log (h_{\theta }(x))-(1-y)\log (1-h_{\theta }(x))$
$J(\theta )$	$-\frac{1}{m}\left[\sum _{i=1}^m{y}^{(i)}\log h_{\theta }(x^{(i)})+(1-y^{(i)})\log (1-h_{\theta }(x^{(i)}))\right]$
${\min }_{\theta }J(\theta )$	重 复 { θ j : = θ j − α 1 m ∑ i = 1 m ( h θ ( x ( i ) ) − y ( i ) ) x j ( i ) } ( j = 0 , … , n ) 重复\{\\\theta_j:=\theta_j-\alpha\frac{1}{m}\sum\limits^m_{i=1}(h_\theta(x^{(i)})-y^{(i)})x^{(i)}_j\\\}\qquad (j=0,\dots,n) 重复{θj​:=θj​−αm1​i=1∑m​(hθ​(x(i))−y(i))xj(i)​}(j=0,…,n)

P.S. Matlab

5 code

function [jVal, gradient] = costFunction(theta)

jVal = (theta(1)-5)*2 + (theta(2)-5)*2;

gradient = zeros(2,1);
gradient(1) = 2*(theta(1)-5);
gradient(2) = 2*(theta(2)-5);

options = optimset('GradObj', 'on', 'MaxIter', '100'); 
initialTheta = zeros(2,1);
[optTheta, functionVal, exitFlag] = fminunc(@costFunction, initialTheta, options)
% optTheta: theta最优解
% functionVal: 代价函数的最小值
% exitFlag: 是否收敛（1为已经收敛）