【机器学习笔记】吴恩达公开课·第七章 Logistic 回归

假设陈述

• h(x) : estimated probability that y = 1 on input x
  $$h_{\theta }(x)=P(y=1|x;\theta )=g({\theta }^{T}x)$$
• sigmod 函数
  $$g(z)=\frac{1}{1+e^{-z}}$$
  

决策界限 Decision Boundary

$$y=\{\begin{array}{ll}1,& h_{\theta }(x)\ge 0.5or{h}_{\theta }(x)\ge 0\\ 0,& h_{\theta }(x)<0.5or{h}_{\theta }(x)<0\end{array}$$

• Non-Linear Decision Boundary
  

Logistic regression cost function

• cost
  $$Cost(h_{\theta }(x^{(i)},y^{(i)}))=\{\begin{array}{l}-\log (h_{\theta }(x))\text{if y = 1}\\ -\log (1-h_{\theta }(x))\text{if y = 0}\end{array}$$
• simplify version

$$Cost(h_{\theta }(x^{(i)},y^{(i)}))=-y\log (h_{\theta }(x))-(1-y)\log (1-h_{\theta }(x))$$

• $J(\theta )$

$$\begin{gathered} J(\theta )=-\frac{1}{m}\sum _{i=1}^{m}Cost(h_{\theta }(x^{(i)},y^{(i)})) \\ =-\frac{1}{m}[\sum _{i=1}^m{y}^{(i)}\log h_{\theta }(x^{(i)})+(1-y^{(i)})\log (1-h_{\theta }(x^{(i)}))] \end{gathered}$$

• to fie parameters $\theta$

$$\underset{\theta }{\min }J(\theta )$$

高级优化

Optimization algorithms:

• Gradient descent
• Conjugate gradient
• BFGS
• L-BFGS

Advantages

• No need to manually pick $\alpha$
• Often faster than gradient descent

Disadvantage

• More complex

Multiclass classification

Train a Logistic regression classifier $h_{\theta }^{(i)}(x)$ for each class $i$ to predict the probability that $y=i$.
On a new input $x$, to maake a prediction, pick the class $i$ that maximizes
$$\underset{i}{\max }{h}_{\theta }^{(i)}(x)$$

$$h_{\theta }^{(i)}(x)=P(y=i|x;\theta )(i=1,2,3)$$