sklearn.SVC参数解读及优化方法

sklearn.SVC官方文档参数解读

SVC文档：
https://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html
默认参数：
class sklearn.svm.SVC(C=1.0, kernel=’rbf’, degree=3, gamma=’auto_deprecated’, coef0=0.0, shrinking=True, probability=False, tol=0.001, cache_size=200, class_weight=None, verbose=False, max_iter=-1, decision_function_shape=’ovr’, random_state=None)

模型介绍：
C-Support Vector Classification.

The implementation is based on libsvm.
The fit time scales at least quadratically二次地 with the number of samples and may be impractical beyond tens of thousands of数万的 samples.
For large datasets consider using sklearn.linear_model.LinearSVC or sklearn.linear_model.SGDClassifier instead, possibly after a sklearn.kernel_approximation.Nystroem transformer.

The multiclass support is handled according to a one-vs-one scheme.
多分类svm 根据OVO策略处理

For details on the precise mathematical formulation of the provided kernel functions and how gamma, coef0 and degree affect each other, see the corresponding section in the narrative叙述的 documentation: Kernel functions.

参数含义：

C : float, optional (default=1.0)；Penalty parameter C of the error term. 分类错误项的惩罚系数，C越大，一旦分类错误，惩罚越严重，C越大，越容易过拟合，越容易欠拟合。

kernel : string, optional (default=’rbf’)；Specifies the kernel type to be used in the algorithm. It must be one of ‘linear’, ‘poly’, ‘rbf’, ‘sigmoid’, ‘precomputed’ or a callable. If a callable is given it is used to pre-compute the kernel matrix from data matrices; that matrix should be an array of shape (n_samples, n_samples).

degree : int, optional (default=3)；Degree of the polynomial kernel function (‘poly’). Ignored by all other kernels.

gamma : float, optional (default=’auto_deprecated’,now)；Kernel coefficient for ‘rbf’, ‘poly’ and ‘sigmoid’.
gamma=‘auto’ :which uses 1 / n_features；
gamma=‘scale’： it uses 1 / (n_features * X.var()) as value of gamma.
gamma= ‘auto_deprecated’： a deprecated version of ‘auto’ is used as a default ， no explicit value of gamma was passed. 'deprecated’弃用的

coef0 : float, optional (default=0.0)
Independent term in kernel function. It is only significant in ‘poly’ and ‘sigmoid’.

shrinking : boolean, optional (default=True)
Whether to use the shrinking heuristic启发式.

probability : boolean, optional (default=False)
Whether to enable probability estimates. This must be enabled prior to calling fit, and will slow down that method.是否保存预测的每一类别的可能性，可以输出，而不仅仅输出预测的类别

tol : float, optional (default=1e-3)
Tolerance容忍 for stopping criterion标准、准则.

cache_size : float, optional，default 200
Specify the size of the kernel cache内核缓存 (in MB).

class_weight : {dict, ‘balanced’}, optional
Set the parameter C of class i to class_weight[i]*C for SVC.
If not given, all classes are supposed to have weight one.
The “balanced” mode uses the values of y to automatically adjust weights inversely proportional to class frequencies in the input data as n_samples / (n_classes * np.bincount(y))

verbose : bool, default: False
Enable verbose详细的 output. Note that this setting takes advantage of a per-process runtime setting in libsvm that, if enabled, may not work properly in a multithreaded context多线程环境.

max_iter : int, optional (default=-1)
Hard limit on iterations within solver, or -1 for no limit.

decision_function_shape : ‘ovo’, ‘ovr’, default=’ovr’；
Whether to return a one-vs-rest (‘ovr’) decision function of shape (n_samples, n_classes) as all other classifiers, or the original one-vs-one (‘ovo’) decision function of libsvm which has shape (n_samples, n_classes * (n_classes - 1) / 2).
However, one-vs-one (‘ovo’) is always used as multi-class strategy.

random_state : int, RandomState instance or None, optional (default=None)
The seed of the pseudo random number伪随机数 generator used when shuffling the data for probability estimates.
If int, random_state is the seed used by the random number generator;
If RandomState instance, random_state is the random number generator;
If None, the random number generator is the RandomState instance used by np.random.
https://www.cnblogs.com/nolonely/p/7007961.html

学习器模型中一般有两个参数：
一类参数可以从数据中学习估计得到;
还有一类参数无法从数据中估计，只能靠人的经验进行指定，后一类参数就叫超参数

比如，支持向量机里的C，Kernel，gama，朴素贝叶斯里的alpha等，在学习其模型的设计中，我们要搜索超参数空间为学习器模型找到最合理的超参数，
可以通过以下方法获得学习器模型的参数列表和当前取值：estimator.get_params()

参数空间的搜索由以下几个部分构成：
1.estimator
2.参数空间（取值范围）
3.搜索/采样方法
4.验证方法
5.评分函数

sklearn 提供了两种通用的参数搜索/采样方法：网络搜索和随机采样，

**网格搜索交叉验证（GridSearchCV）：**以穷举的方式遍历所有可能的参数组合
**随机采样交叉验证（RandomizedSearchCV）：**依据某种分布对参数空间采样，随机的得到一些候选参数组合方案
sklearn.model_selection：
GridSearchCV,
RandomizedSearchCV,
ParameterGrid,
ParameterSampler,
fit_grid_point
详见上面网页

GridSearchCV

穷举所有候选参数组合，一个组合一个组合试，直到找到最好的

param_grid=[

{‘C’:[1,10,100,1000],‘kernel’:[‘linear’]},

{‘C’:[1,10,100,1000],‘gamma’:[0.001,0.0001],‘kernel’:[‘rbf’]}

]

上面的参数指定了要搜索的两个网格（每个网格就是一个字典，不同网格内分别搜索，不交叉）：
第一个里面有4×1个候选参数组合，第二个里面有4×2×1=8个候选参数组合

GridSearchCV的实例实现了通用的estimator API（其他学习模型也可以使用）：当在数据集上训练的时候，所有可能的参数组合将会被评估，训练完成后选组最优的参数组合对应的estimator。#print(estimator.best_estimator_)

from sklearn import svm,datasets
from sklearn.model_selection import GridSearchCV

iris=datasets.load_iris()
parameters={'kernel':('rbf','linear'),'C':[1,5,10]}
svr=svm.SVC()
clf=GridSearchCV(svr,parameters)
clf.fit(iris.data,iris.target)
print(clf.best_estimator_)
#输出：
SVC(C=1, cache_size=200, class_weight=None, coef0=0.0,
  decision_function_shape=None, degree=3, gamma='auto', kernel='linear',
  max_iter=-1, probability=False, random_state=None, shrinking=True,
  tol=0.001, verbose=False)

https://blog.csdn.net/lujiandong1/article/details/46386201
SVM模型有两个非常重要的参数C与gamma。
C是惩罚系数，即对误差的宽容度。c越高，说明越不能容忍出现误差,容易过拟合。C越小，容易欠拟合。C过大或过小，泛化能力变差
gamma是选择RBF函数作为kernel后，该函数自带的一个参数。隐含地决定了数据映射到新的特征空间后的分布，gamma越大，支持向量越少，gamma值越小，支持向量越多。支持向量的个数影响训练与预测的速度。