python评分卡4_logistics原理与解法_sklearn英译汉

本系列分以下章节：
python评分卡1_woe与IV值
python评分卡2_woe与IV分箱方法
python评分卡3_woe与IV分箱实现
python评分卡4_logistics原理与解法_sklearn英译汉
python评分卡5_Logit例1_plot_logistic_l1_l2_sparsity
python评分卡6_Logit例2plot_logistic_path

logistics_sklearn网址

Logistic regression

Logistic regression, despite its name, is a linear model for classification rather than regression. Logistic regression is also known in the literature as logit regression, maximum-entropy classification (MaxEnt) or the log-linear classifier. In this model, the probabilities describing the possible outcomes of a single trial are modeled using a logistic function.

Logistic regression is implemented in LogisticRegression. This implementation can fit binary, One-vs-Rest, or multinomial logistic regression with optional or l1,l2, Elastic-Net regularization.

逻辑回归，尽管它的名字，是一个线性模型的分类，而不是回归。 Logistic回归在文献中也称为logit回归、最大熵分类（MaxEnt）或对数线性分类器。
在该模型中，描述单个试验可能结果的概率使用逻辑函数建模。
Logistic回归是在 LogisticRegression 中实现的,这个实现可以适应二分类，一对剩余，优化的多项式逻辑回归，带有正则的l1,l2或者弹性网

Note Regularization is applied by default, which is common in machine learning but not in statistics. Another advantage of regularization is that it improves numerical stability. No regularization amounts to setting C to a very high value.

注意：默认情况下会应用正则化，它在机器学习中很常见，但是在统计学中并不常见。正则化的另一个优点是提高了数值稳定性。没有正则化等于将C设置为非常高的值。

As an optimization problem, binary class l2 penalized logistic regression minimizes the following cost function:
作为一个优化问题，二分类采用l2惩罚因子的逻辑回归最小化以下损失函数：
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-pgQNOnY9-1652495571393)(attachment:image.png)]

Similarly,l1 regularized logistic regression solves the following optimization problem:
同样地，采用l1正则的逻辑回归解决以下优化问题：
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-YpOeCI3H-1652495571396)(attachment:image.png)]

Elastic-Net regularization is a combination of l1 and l2 , and minimizes the following cost function:
弹性网正则结合了l1与l2正则，最小化以下损失函数：
[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-A7rRQ68w-1652495571398)(attachment:image.png)]

where P controls the strength of l1 regularization vs.l2 regularization (it corresponds to the l1_ratio parameter).
Note that, in this notation, it’s assumed that the target yi takes values in the set -1,1 at trial i . We can also see that Elastic-Net is equivalent to l1 when P=1 and equivalent to l2 when P=0 .
这里P控制l1正则相对于L2正则的强度，(它相当于参数 l1_ratio）
注意 在这个公式中，它假设对于样本i,目标值yi只取集合-1,1中的值。我们还可以看到，当P=1时，弹性网等价于l1正则，当P=0时，弹性网等价于L2正则。

The solvers implemented in the class LogisticRegression are “liblinear”, “newton-cg”, “lbfgs”, “sag” and “saga”:
LogisticRegression类中实现的求解算法有“liblinear”、“newton cg”、“lbfgs”、“sag”和“saga”：

The solver “liblinear” uses a coordinate descent (CD) algorithm, and relies on the excellent C++ LIBLINEAR library, which is shipped with scikit-learn. However, the CD algorithm implemented in liblinear cannot learn a true multinomial (multiclass) model; instead, the optimization problem is decomposed in a “one-vs-rest” fashion so separate binary classifiers are trained for all classes. This happens under the hood, so LogisticRegression instances using this solver behave as multiclass classifiers. For l1
regularization sklearn.svm.l1_min_c allows to calculate the lower bound for C in order to get a non “null” (all feature weights to zero) model.

算法（解法）“liblinear”使用坐标下降（CD）算法，并依赖于scikit-learn附带的优秀C++LIBLINEAR库。然而，在liblinear中实现的CD算法无法学习真正的多项式（多类）模型；取而代之的是，优化问题以“一对多”的方式进行分解，所有的类别都以二分类的方式训练。这发生在引擎下，因此LogisticRegression实例 使用该算法解决多分类。对于l1 正则项， sklearn.svm.l1_min_c 用来计算C的下限，以获得非“空”（所有特征权重为零）模型。

The “lbfgs”, “sag” and “newton-cg” solvers only support L2 regularization or no regularization, and are found to converge faster for some high-dimensional data. Setting multi_class to “multinomial” with these solvers learns a true multinomial logistic regression model 5, which means that its probability estimates should be better calibrated than the default “one-vs-rest” setting.

算法（解法）“lbfgs”、“sag” 和"newton cg"仅支持L2正则化或非正则化，对于某些高维数据，它们的收敛速度更快。采用这种解法将 multi_class 设置成 “multinomial” 将学习一个真正的多项式逻辑回归模型，这意味着它的概率估计应该比默认的“one-vs-rest”设置校准地更好。

The “sag” solver uses Stochastic Average Gradient descent 6. It is faster than other solvers for large datasets, when both the number of samples and the number of features are large.

算法（解法）“sag” 使用随机平均梯度下降 6.当样本数和特征数都较大时，它比其他大型数据集的算法（解法）更快.

The “saga” solver 7 is a variant of “sag” that also supports the non-smooth penalty=“l1”. This is therefore the solver of choice for sparse multinomial logistic regression. It is also the only solver that supports penalty=“elasticnet”.

算法（解法） “saga” 是 “sag”的改造，也支持非平滑“l1”惩罚。因此这是稀疏多项式逻辑回归的首选算法（解法）器。它也是唯一支持惩罚=“elasticnet”的算法（解法）器。

The “lbfgs” is an optimization algorithm that approximates the Broyden–Fletcher–Goldfarb–Shanno algorithm 8, which belongs to quasi-Newton methods. The “lbfgs” solver is recommended for use for small data-sets but for larger datasets its performance suffers. 9

“lbfgs”是一种优化算法，近似于Broyden–Fletcher–Goldfarb–Shanno算法{8}，属于拟牛顿法。“lbfgs”解算器建议用于小型数据集，但对于大型数据集，其性能会受到影响。[9]

The following table summarizes the penalties supported by each solver:
下表总结了每个解算器支持的惩罚：

[外链图片转存失败,源站可能有防盗链机制,建议将图片保存下来直接上传(img-2Hdhzrku-1652495571398)(attachment:image.png)]

The “lbfgs” solver is used by default for its robustness. For large datasets the “saga” solver is usually faster. For large dataset, you may also consider using SGDClassifier with ‘log’ loss, which might be even faster but requires more tuning.

默认情况下，使用“lbfgs”解法，因其比其他解法更具有健壮性。对于大型数据集，“saga”解法通常更快。对于大型数据集，您还可以考虑使用带有“log”损失的SGDClassizer，这可能更快，但需要更多的调整。

Note Feature selection with sparse logistic regression

A logistic regression with l1 penalty yields sparse models, and can thus be used to perform feature selection, as detailed in L1-based feature selection.

注：稀疏逻辑回归的特征选择

具有l1惩罚的逻辑回归产生稀疏模型，因此可用于执行特征选择，详情见基于L1的特征选择中所述。

LogisticRegressionCV implements Logistic Regression with built-in cross-validation support, to find the optimal C and l1_ratio parameters according to the scoring attribute. The “newton-cg”, “sag”, “saga” and “lbfgs” solvers are found to be faster for high-dimensional dense data, due to warm-starting (see Glossary).

LogisticRegressionCV实现支持内置交叉验证的Logistic回归，根据评分属性找到最佳的C和l1_ratio参数。由于热启动，对于高维密集数据“newton cg”、“sag”、“saga”和“lbfgs”解法器更快。