正则化是如何影响分类质量的呢?我们使用吴恩达机器学习课程中的「微芯片测试」数据集,运用基于多项式特征的逻辑回归方法,然后改变正则化参数 C" role="presentation" style=" box-sizing: border-box; display: inline-table; line-height: normal; font-size: 16px; text-align: left; overflow-wrap: normal; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; min-height: 0px; border-width: 0px; border-style: initial; border-color: initial; color: rgb(0, 0, 0); font-family: "Helvetica Neue", Helvetica, Arial, sans-serif; background-color: rgb(255, 255, 255); ">?C。首先,看看正则化是如何影响分类器的分界,并查看欠拟合和过拟合的情况。
参数 C" role="presentation" style=" box-sizing: border-box; display: inline-table; line-height: normal; word-spacing: normal; overflow-wrap: normal; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; min-height: 0px; border-width: 0px; border-style: initial; border-color: initial; ">?C 越大,模型中可恢复的数据之间的关系就越复杂(直观地说,C" role="presentation" style=" box-sizing: border-box; display: inline-table; line-height: normal; word-spacing: normal; overflow-wrap: normal; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; min-height: 0px; border-width: 0px; border-style: initial; border-color: initial; ">?C 对应模型的「复杂度」:模型能力)。
如果正则化过强,即 C" role="presentation" style=" box-sizing: border-box; display: inline-table; line-height: normal; word-spacing: normal; overflow-wrap: normal; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; min-height: 0px; border-width: 0px; border-style: initial; border-color: initial; ">?C 值很小,最小化逻辑损失函数问题的解(权重)可能过小或为零。这样的模型对误差的「惩罚」也不够(即在上面的函数 J(X,y,w)" role="presentation" style=" box-sizing: border-box; display: inline-table; line-height: normal; word-spacing: normal; overflow-wrap: normal; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; min-height: 0px; border-width: 0px; border-style: initial; border-color: initial; ">?(?,?,?)J(X,y,w) 中,权重的平方和过高,导致误差 L" role="presentation" style=" box-sizing: border-box; display: inline-table; line-height: normal; word-spacing: normal; overflow-wrap: normal; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; min-height: 0px; border-width: 0px; border-style: initial; border-color: initial; ">?L 较大)。在这一情形下,模型将会欠拟合。
如果正则化过弱,即 C" role="presentation" style=" box-sizing: border-box; display: inline-table; line-height: normal; word-spacing: normal; overflow-wrap: normal; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; min-height: 0px; border-width: 0px; border-style: initial; border-color: initial; ">?C 值很大,最小化逻辑损失函数问题的解可能权重过大。在这一情形下, L" role="presentation" style=" box-sizing: border-box; display: inline-table; line-height: normal; word-spacing: normal; overflow-wrap: normal; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; min-height: 0px; border-width: 0px; border-style: initial; border-color: initial; ">?L 对函数 J(X,y,w)" role="presentation" style=" box-sizing: border-box; display: inline-table; line-height: normal; word-spacing: normal; overflow-wrap: normal; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; min-height: 0px; border-width: 0px; border-style: initial; border-color: initial; ">?(?,?,?)J(X,y,w) 贡献较大,导致过拟合。
C" role="presentation" style=" box-sizing: border-box; display: inline-table; line-height: normal; word-spacing: normal; overflow-wrap: normal; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; min-height: 0px; border-width: 0px; border-style: initial; border-color: initial; ">?C 是一个超参数,逻辑回归不会自动学习调整 C" role="presentation" style=" box-sizing: border-box; display: inline-table; line-height: normal; word-spacing: normal; overflow-wrap: normal; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; min-height: 0px; border-width: 0px; border-style: initial; border-color: initial; ">?C 的值,我们可以通过交叉验证来人工选择较好的 C" role="presentation" style=" box-sizing: border-box; display: inline-table; line-height: normal; word-spacing: normal; overflow-wrap: normal; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; min-height: 0px; border-width: 0px; border-style: initial; border-color: initial; ">?C 值。