Hands-On Machine Learning with Scikit-Learn & TensorFlow Exercise Q&A Chapter04

最新推荐文章于 2021-07-06 20:47:05 发布

Leonardo Liu

最新推荐文章于 2021-07-06 20:47:05 发布

阅读量492

点赞数 1

分类专栏： Python 机器学习 Scikit-Learn Hands-On ML with sklearn & TensorFlow Exercise Q&A 文章标签： Machine Learning HandsOn

本文链接：https://blog.csdn.net/leowinbow/article/details/88700816

版权

本章节主要探讨了在大规模特征训练集上适用的线性回归算法（如随机梯度下降、小批量梯度下降），以及特征尺度不一可能对算法造成的影响。此外，解释了梯度下降算法在逻辑回归中的行为，以及不同梯度下降算法在收敛速度和模型质量上的差异。针对过拟合和欠拟合问题，提出了正则化、调整模型复杂度和增大训练集等解决方案。同时，讨论了多项式回归、岭回归、拉索回归和弹性网络等在防止过拟合方面的应用。最后，指出在二分类与多分类任务中如何选择逻辑回归或softmax回归。

摘要由CSDN通过智能技术生成

Q1. What Linear Regression training algorithm can you use if you have a training set with millions of features?

A1:Stochastic Gradient Descent or Mini-Batch Gradient Descent.

Q2. Suppose the features in your training set have very different scales. What algorithms might suffer from this, and how? What can you do about it?

A2: The normal Function will work well. We can scale the training set before training the model, use StandardScaler class.

Q3. Can Gradient Descent get stuck in a local minimum when training a Logistic Rregression model?

A3: Logistic Rregression's cost function is convex, so it cannot get stuck in a local minimum.

Q4. Do all Gradient Descent algorithms lead to the same model provided you let them run long enough?

A4: Theoratically, if the optimization problem is convex, all Gradient Descent algorithms will approach the global optimum and end up producing fairly similiar models. But unless you gradually reduce the learning rate, Stochastic GD and Mini-Batch GD will never truly converge, because they will keep jumping back and forth around the global optimum without reaching it, even if you let them run long enough. So they will end up producing slightly different.

Q5. Suppose you use Batch Gradient Descent and you plot the validation error at every epoch. If you notice that the validation error consistently goes up, what is likely going on? How can you fix this?

A5: The learning rate is too high, and the model is diverging. Or you have overfitting the training set. We need to check the training error, if training error continues to go down, it's overfitting, we need to plus regularization or change to a more simple model; if training error also goes up, that means the learning rate is too high, we need to reduce the learninig rate.

Q6. Is it a good idea to stop Mini-Batch Gradient Descent immediately when the validation error goes up?

A6: No, because it's too early, the Mini-Batch GD is not guaranteed to make progress every epoch. We can save the model at regular intervals, and when it has not improved for a really long time, then we can stop it can come back the best sav