Andrew Ng Machine Learning 第十周
前言
网易云课堂(双语字幕,不卡):https://study.163.com/course/courseMain.htm?courseId=1004570029
Coursera:https://www.coursera.org/learn/machine-learning
本人初学者,先在网易云课堂上看网课,再去Coursera上做作业,开博客以记录,文章中引用图片皆为课程中所截。
大规模机器学习
1.学习大数据集
Tips:按照以前的算法计算,后面的累加项处若m十分之大,计算量将会很大
2.随机梯度下降
Tips:先将训练集随机化,定义一个cost函数,对每组单组计算代价函数的J(θ)min,每次计算只需用一对(x(i),y(i)),用这一组来对θj梯度下降,其中α后面的是cost函数的导数
Tips:在图上看它将会对每一组进行梯度下降,也就是靠近,最后将会在一定范围内变化,其中有全局最小值
3.Mini-Batch 梯度下降
Tips:相比于随机梯度下降的每组样本,mini-batch梯度下降采用的是b组一起(其中b由自己选定,一般为10-100)
4.随机梯度下降收敛
Tips:每1000个迭代后计算一次平均值,画图判断是否收敛
Tips:该算法中的学习率α
5.在线学习
Tips:x为一个用户的特征,y=1即为会选择使用该项服务,我们的在线学习即是每得到一对(x,y),就更新θ参数,使得它参数越来越接近最新情况
Tips:当用户进行一个搜索时,计算出该用户点击率最高的几个选项来提供
6.减少映射与数据并行
Tips:简单来说,假设它训练集有400个特征,将400个分成n份,分给n个不同的电脑进行计算,等于每个电脑只算了大工作量的1/4,最后再结合计算
题目
1.Question 1
Suppose you are training a logistic regression classifier using stochastic gradient descent. You find that the cost (say, cost(θ,(x(i),y(i))), averaged over the last 500 examples), plotted as a function of the number of iterations, is slowly increasing over time. Which of the following changes are likely to help?
解答:D
2.Question 2
Which of the following statements about stochastic gradient
descent are true? Check all that apply.
解答:BC
3.Question 3
Which of the following statements about online learning are true? Check all that apply.
解答:BC
4.Question 4
Assuming that you have a very large training set, which of the
following algorithms do you think can be parallelized using
map-reduce and splitting the training set across different
machines? Check all that apply.
解答:AD
(对于BC,map-reduce基本用在大型梯度下降的分解)
5.Question 5
Which of the following statements about map-reduce are true? Check all that apply.
解答:ABC