Coursera | Andrew Ng (03-week1-1.12)—改善你的模型的表现

该系列仅在原课程基础上部分知识点添加个人学习笔记，或相关推导补充等。如有错误，还请批评指教。在学习了 Andrew Ng 课程的基础上，为了更方便的查阅复习，将其整理成文字。因本人一直在学习英语，所以该系列以英文为主，同时也建议读者以英文为主，中文辅助，以便后期进阶时，为学习相关领域的学术论文做铺垫。- ZJ

Coursera 课程 |deeplearning.ai |网易云课堂

---

转载请注明作者和出处：ZJ 微信公众号-「SelfImprovementLab」

知乎：https://zhuanlan.zhihu.com/c_147249273

CSDN：http://blog.csdn.net/junjun_zhao/article/details/79159515

---

1.12 Improving your model performance (改善你的模型的表现)

(字幕来源：网易云课堂)

You have heard about orthogonalization.How to set up your dev and test sets,human level performance as a proxy for Bayes’s error and how to estimate your avoidable bias and variance.Let’s pull it all together into a set of guidelines for how to improve the performance of your learning algorithm.So, I think getting a supervised learning algorithm to work well means fundamentally hoping or assuming that you can do two things.First is that you can fit the training set pretty well and you can think of this as roughly saying that you can achieve low avoidable bias.And the second thing you’re assuming can do well is that doing well in the training set generalizes pretty well to the dev set or the test setand this is sort of saying that variance is not too bad.And in the spirit of orthogonization,what you see is that there’s a second set of knobs to fix the avoidable bias issues such as training a bigger network or training longer.

你们学过正交化，如何设立开发集和测试集，用人类水平错误率来估计贝叶斯错误率，以及如何估计可避免偏差和方差，我们现在把它们全部组合起来 写成一套指导方针，如何提高学习算法性能的指导方针，所以我想要让一个监督学习算法达到实用 基本上，希望或者假设你可以完成两件事情，首先 你的算法对训练集的拟合很好，这可以看成是，你能做到可避免偏差很低，还有第二件事你可以做好的是，在训练集中做得很好，然后推广到开发集和测试集也很好，这就是说方差不是太大，在正交化的精神下，你可以看到这里有第二组旋钮 可以修正可避免偏差问题，比如训练更大的网络 或者训练更久。

And there’s a separate set of things you can use to address variance problems,such as regularization or getting more training data.So to summarize of the process we’ve seen in the last several videos,if you want to improve the performance of your machine learning system,I would recommend looking at the difference between your training error and your proxy for bayes error and this gives you a sense of the avoidable bias.In other words, just how much better do you think you should be trying to do on your training set and then look at the difference between your dev error and your training error as an estimate.So, it’s how much of a variance problem you have.In other words,how much harder you should be working to make your performance generalize from the training set to the dev set,that it wasn’t trained on explicitly?So to whatever extent you want to try to reduce avoidable bias,I would try to apply tactics like train a bigger model.So, you can just do better on your training sets or train longer.Use a better optimization algorithm such as.Adds momentum or RMS prop,or use a better algorithm like Adam.Or one of the thing you could try is to just find a better new neural network architecture or better said, hyperparameters and this could include everything from changing the activation functions to changing the number of layers or hidden units.Although you do that, it would be in the direction of increasing the model size to try out other models or other models architectures,such as Recurrent neural network and convolutional neural networks which we’ll see in later courses.Whether or not a new neural network architecture willfit your training set better is sometimes hard to tell in advance,but sometimes you can get much better results with a better architecture.

还有一套独立的技巧可以用来处理方差问题，比如正则化或者收集更多训练数据，总结一下前几段视频我们见到的步骤，如果你想提升机器学习系统的性能，我建议你们看看训练错误率，和贝叶斯错误率估计值之间的距离 让你知道可避免偏差有多大，换句话说 就是你觉得还能做多好，你对训练集的优化还有多少空间 然后看看，你的开发错误率和训练错误率之间的距离，就知道你的方差问题有多大，换句话说，你应该做多少努力，让你的算法表现能够从训练集推广到开发集，算法是没有在开发集上训练的，如果你想用尽一切办法减少可避免偏差，我建议试试这样的策略 比如使用规模更大的模型，这样算法在训练集上的表现会更好 或者训练更久，使用更好的优化算法 比如说，加入 momentum 或者 RMSprop，或者使用更好的算法 比如 Adam，你还可以试试，寻找更好的新神经网络架构，或者说 更好的超参数，这些手段包罗万有 你可以改变激活函数，改变层数 或者隐藏单位数，虽然你这么做可能会让模型规模变大，或者试用其他模型 其他架构，如循环神经网络和卷积神经网络，在之后的课程里我们会详细介绍的，新的神经网络架构能否，更好地拟合你的训练集 有时也很难预先判断，但有时换架构可能会得到好得多的结果。

Next to the extent that you find out variance is a problem.Some of the many of the techniques you could try, then includes the following.You can try to get more data, because getting more data to train on could help you generalize better to dev set data that you didn’t see.You could try regularization.So this includes things like L2 regularization or dropout,or data augmentation which we had talk about in the previous course.Or once again, you can also try various neural network architecture,hyperparameters search to see if that can help you find a neural network architecture that is better suited for your problem.I think that this notion of bias or avoidable bias and variance there is one of those things that easily learned, but tough to master and if you are able to systematically apply the concept from this week’s videos.You actually be much more efficient and much more systematic and much more strategic than a lot of machine learning teams in terms of how tosy stematically go about improving the performance of their machine learning system.So, that this week’s whole work will allow you to practice andexercise more your understanding of these concepts.Best of luck with this week’s homework and I look forward to also seeing you in next week’s videos.

另外当你发现方差是个问题时，你可以试用很多技巧 包括以下这些，你可以收集更多数据 因为收集更多数据去训练，可以帮你更好地推广到系统看不到的开发集数据，你可以尝试正则化，包括 L2 正则化 Dropout 正则化，或者我们在之前课程中提到的数据增强，同时你也可以试用不同的神经网络架构，超参数搜索 看看能不能帮助你，找到一个更适合你的问题的神经网络架构，我想这些偏差 可避免偏差 和方差的概念，是容易上手 难以精通的，如果你能系统全面地应用本周课程里的概念，你实际上会比很多现有的机器学习团队，更有效率 更系统 更有策略地，系统提高机器学习系统的性能，所以 本周的作业中 你们可以练习一下，更深入地理解这些概念，祝你们好运，我希望下周的视频中再见到你们。

---

重点总结：

改善模型的表现

基本假设：

• 模型在训练集上有很好的表现；
• 模型推广到开发和测试集啥会给你也有很好的表现。

减少可避免偏差

• 训练更大的模型 （多层的网络，或 每层网络中更多的隐藏单元）
• 训练更长时间、训练更好的优化算法（Momentum、RMSprop、Adam）
• 寻找更好的网络架构（RNN、CNN）、寻找更好的超参数

减少方差

• 收集更多的数据
• 正则化（L2、dropout、数据增强）
• 寻找更好的网络架构（RNN、CNN）、寻找更好的超参数

参考文献：

[1]. 大树先生.吴恩达Coursera深度学习课程 DeepLearning.ai 提炼笔记（3-1）– 机器学习策略（1）

---

PS: 欢迎扫码关注公众号：「SelfImprovementLab」！专注「深度学习」，「机器学习」，「人工智能」。以及 「早起」，「阅读」，「运动」，「英语 」「其他」不定期建群 打卡互助活动。