正则化 交叉验证

Regularization is common in machine learning but not in statistics. Another advantage of regularization is that it improves numerical stability.机器学习中正则化是非常普遍的在使用，但在统计学，数据科学中用的不多，它可以提高模型的数值稳定性。

机器学习中的范数规则化之L0、L1与L2范数

知识点 | 关于机器学习的超全总结

看过的最好的讲训练集验证集测试集的文章！
如何正确使用机器学习中的训练集、验证集和测试集？

这个图完美了

不正则化的线性回归经过：
L1正则化-------LASSO回归 least absolute shrinkage and selection operator
L2正则化-------ridge回归
L1+L2 正则化 elastic net 回归
elastic net is a linear regression model trained with both and -norm regularization of the coefficients.

Elastic-net is useful when there are multiple features which are correlated with one another. Lasso is likely to pick one of these at random, while elastic-net is likely to pick both.

A practical advantage of trading-off between Lasso and Ridge is that it allows Elastic-Net to inherit some of Ridge’s stability under rotation.

线性回归：

岭回归：

lasso回归：

elastic net回归：
正则化项是L1和L2的凸组合，Convex combination of L2 and L1

ElasticNet算法解析