Stanford ML - Lecture 11 - Large scale machine learning

最新推荐文章于 2019-07-25 12:09:00 发布

Quebradawill

最新推荐文章于 2019-07-25 12:09:00 发布

阅读量779

点赞数

分类专栏： ML-Stanford-Andrew Ng Machine Learning

版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。

本文链接：https://blog.csdn.net/qiudw/article/details/8702580

版权

Machine Learning 同时被 2 个专栏收录

19 篇文章 0 订阅

订阅专栏

ML-Stanford-Andrew Ng

12 篇文章 0 订阅

订阅专栏

1. Learning with large datasets

It's not who has the best algorithm that wins. It's who has the most data

2. Stochastic gradient descent

batch gradient descent

$J_{train}(\theta) = \frac{1}{2m} \sum_{i=1}^m \left( h_{\theta}(x^{(i)} - y^{(i)}) \right )^2$

repeat

$\theta_j := \theta_j - \alpha \frac{1}{m} \sum_{i=1}^m \left( h_{\theta}(x^{(i)}) - y^{(i)} \right ) x_j^{(i)}$

$(\textrm{for every} \ j = 0, \cdots, n)$

stochastic gradient descent

$cost(\theta, (x^{(i)}, y^{(i)})) = \frac{1}{2} \left( h_{\theta}(x^{(i)}) - y^{(i)} \right )^2$

$J_{train}(\theta) = \frac{1}{m} \sum_{i=1}^m cost(\theta, (x^{(i)}, y^{(i)}))$

randomly shuffle datasets
repeat

$\textrm{for} \ i = 1, \cdots, m$

$\theta_j := \theta_j - \alpha \left( h_{\theta}(x^{(i)}) - y^{(i)} \right ) x_j^{(i)}$

$(\textrm{for every} \ j = 0, \cdots, n)$

3. Mini-batch gradient descent

batch gradient descent: use all $m$ examples in each iteration
stochastic gradient descent: use all 1 example in each iteration
mini-batch gradient descent: use all $b$ examples in each iteration

4. Stochastic gradient descent convergence

Learning rate $\alpha$ is typically held constant. Can slowly decrease $\alpha$ over time if we want $\theta$ to converge

5. Online learning

6. Map-reduce and data parallelism

关注

0
点赞
踩
0

收藏

觉得还不错? 一键收藏
0
评论
复制链接

分享到 QQ

分享到新浪微博

扫一扫

专栏目录

评论

被折叠的条评论为什么被折叠?

到【灌水乐园】发言

查看更多评论

添加红包

成就一亿技术人!

hope_wisdom

发出的红包

实付元

使用余额支付

点击重新获取

扫码支付

钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。