float js 正则 验证_逻辑回归正则化实例

正则化是如何影响分类质量的呢？我们使用吴恩达机器学习课程中的「微芯片测试」数据集，运用基于多项式特征的逻辑回归方法，然后改变正则化参数 C" role="presentation" style=" box-sizing: border-box; display: inline-table; line-height: normal; font-size: 16px; text-align: left; overflow-wrap: normal; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; min-height: 0px; border-width: 0px; border-style: initial; border-color: initial; color: rgb(0, 0, 0); font-family: "Helvetica Neue", Helvetica, Arial, sans-serif; background-color: rgb(255, 255, 255); ">?C。首先，看看正则化是如何影响分类器的分界，并查看欠拟合和过拟合的情况。

• 参数 C" role="presentation" style=" box-sizing: border-box; display: inline-table; line-height: normal; word-spacing: normal; overflow-wrap: normal; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; min-height: 0px; border-width: 0px; border-style: initial; border-color: initial; ">?C 越大，模型中可恢复的数据之间的关系就越复杂(直观地说，C" role="presentation" style=" box-sizing: border-box; display: inline-table; line-height: normal; word-spacing: normal; overflow-wrap: normal; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; min-height: 0px; border-width: 0px; border-style: initial; border-color: initial; ">?C 对应模型的「复杂度」：模型能力)。

• 如果正则化过强，即 C" role="presentation" style=" box-sizing: border-box; display: inline-table; line-height: normal; word-spacing: normal; overflow-wrap: normal; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; min-height: 0px; border-width: 0px; border-style: initial; border-color: initial; ">?C 值很小，最小化逻辑损失函数问题的解(权重)可能过小或为零。这样的模型对误差的「惩罚」也不够(即在上面的函数 J(X,y,w)" role="presentation" style=" box-sizing: border-box; display: inline-table; line-height: normal; word-spacing: normal; overflow-wrap: normal; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; min-height: 0px; border-width: 0px; border-style: initial; border-color: initial; ">?(?,?,?)J(X,y,w) 中，权重的平方和过高，导致误差 L" role="presentation" style=" box-sizing: border-box; display: inline-table; line-height: normal; word-spacing: normal; overflow-wrap: normal; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; min-height: 0px; border-width: 0px; border-style: initial; border-color: initial; ">?L 较大)。在这一情形下，模型将会欠拟合。

• 如果正则化过弱，即 C" role="presentation" style=" box-sizing: border-box; display: inline-table; line-height: normal; word-spacing: normal; overflow-wrap: normal; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; min-height: 0px; border-width: 0px; border-style: initial; border-color: initial; ">?C 值很大，最小化逻辑损失函数问题的解可能权重过大。在这一情形下， L" role="presentation" style=" box-sizing: border-box; display: inline-table; line-height: normal; word-spacing: normal; overflow-wrap: normal; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; min-height: 0px; border-width: 0px; border-style: initial; border-color: initial; ">?L 对函数 J(X,y,w)" role="presentation" style=" box-sizing: border-box; display: inline-table; line-height: normal; word-spacing: normal; overflow-wrap: normal; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; min-height: 0px; border-width: 0px; border-style: initial; border-color: initial; ">?(?,?,?)J(X,y,w) 贡献较大，导致过拟合。

• C" role="presentation" style=" box-sizing: border-box; display: inline-table; line-height: normal; word-spacing: normal; overflow-wrap: normal; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; min-height: 0px; border-width: 0px; border-style: initial; border-color: initial; ">?C 是一个超参数，逻辑回归不会自动学习调整 C" role="presentation" style=" box-sizing: border-box; display: inline-table; line-height: normal; word-spacing: normal; overflow-wrap: normal; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; min-height: 0px; border-width: 0px; border-style: initial; border-color: initial; ">?C 的值，我们可以通过交叉验证来人工选择较好的 C" role="presentation" style=" box-sizing: border-box; display: inline-table; line-height: normal; word-spacing: normal; overflow-wrap: normal; float: none; direction: ltr; max-width: none; max-height: none; min-width: 0px; min-height: 0px; border-width: 0px; border-style: initial; border-color: initial; ">?C 值。