机器学习-Week 3

最新推荐文章于 2020-03-08 16:25:43 发布

SCP3994

最新推荐文章于 2020-03-08 16:25:43 发布

阅读量160

点赞数

分类专栏：课程笔记文章标签： Coursera Machine Learning

本文链接：https://blog.csdn.net/Rowena_Lee/article/details/82527879

版权

课程笔记专栏收录该内容

1 篇文章 0 订阅

订阅专栏

分类算法-逻辑回归

Logistic (sigmoid) function:

$P(y=1|x;\theta)=h_\theta(x)=\frac{1}{1+e^{-\theta^Tx}} \in[0,1]$

$P(y=0|x;\theta)+P(y=1|x;\theta)=1$

Cost function

$J(\theta)=\frac{1}{m}\sum_{i=1}^m\textrm{Cost}(h_\theta(x)^{(i)},y^{(i)})$

$\textrm{Cost}(h_\theta(x),y)=\left\{\begin{matrix} -\log (h_\theta(x)) & y=1\\ -\log (1-h_\theta(x)) & y=0 \end{matrix}\right.$
$\Rightarrow J(\theta)=-\frac{1}{m}\sum_{i=1}^m[y^{(i)}\log(h_\theta(x^{(i)}))+ (1-y^{(i)})\log(1-h_\theta(x^{(i)}))]$

Iteration: the form is the same as linear regression

$\theta_j=\theta_j-\frac{\alpha}{m}\sum_{i=1}^m(h_\theta(x^{(i)})-y^{(i)})x_j^{(i)}$

Decision boundary：
That's a property of the hypothesis and of the parameters of the hypothesis and not a property of the data set.
Predict when $\theta^Tx\ge 0(h_\theta(x)\ge0.5)$ ; predict when $\theta^Tx\le0(h_\theta(x)<0.5)$ .
Multiclass Classification: one-vs-all method

注意事项：

{0（negative class），1（positive class）}
不适用线性回归

欠拟合与过拟合

Two main options to address the issue of overfitting:

Reduce the number of features:

Manually select which features to keep.
Use a model selection algorithm (studied later in the course).

2. Regularization

Keep all the features, but reduce the magnitude of parameters $\theta_j$
Regularization works well when we have a lot of slightly useful features.

$\min_\theta\frac{1}{2m}\sum_{i=1}^m(h_\theta(x^{(i)})-y^{(i)})^2+\lambda\sum_{j=1}^n\theta_j^2$

where $\lambda$ (palarization parameter) should be adequately large.

Gradient descent

$\theta_j=\theta_j(1-\alpha\frac{\lambda}{m})-\frac{\alpha}{m}\sum_{i=1}^m(h_\theta(x^{(i)})-y^{(i)})x_j^{(i)}$

Cost function

$J(\theta)=-\frac{1}{m}\sum_{i=1}^m[y^{(i)}\log(h_\theta(x^{(i)}))+ (1-y^{(i)})\log(1-h_\theta(x^{(i)}))]+\frac{\lambda}{2m}\sum_{j=1}^n\theta_j^2$

notice that $\theta_0$ is excluded!

SCP3994

关注

0
点赞
踩
0

收藏

觉得还不错? 一键收藏
0
评论
机器学习-Week 3

分类算法-逻辑回归Logistic (sigmoid) function: Cost function Iteration: the form is the same as linear regression Decision boundary： That's a property of...
复制链接

扫一扫