一、数据批量预处理之Pipeline类
Pipeline可以将特征化、标准化等处理合在一起形成一个典型的机器学习问题工作流。即简化了代码还可以结合GridSearchCV进行最佳参数选取。Pipeline传入的是一个包含多个元组的列表,元组第一个是自定义名,后面是算法函数。
from sklearn.pipeline import Pipeline
pipeline=Pipeline([('aaa',TfidVectorizer()),('bbb',LogisticRegression())])
pipeline.fit(x,y)
pipeline.predict(x)
二、最佳参数选取之GridSearchCV类
GridSearchCV类接受一个包含所有被微调的超参数集合。并评估该集合的笛卡尔乘积的每一个元素上的模型的性能。
from sklearn.pipeline import Pipeline
from sklearn.model_selection import GridSearchCV
pipeline=Pipeline([('aaa',TfidVectorizer()),('bbb',LogisticRegression())])
parameters={'aaa__max_df':(0.25,0.5,0.75),'bbb__C':(0.01,0.1,1,10)}
grid_search = GridSearchCV(pipeline,parameters,n_jobs=2,verbose=1,scoring='accuracy',cv=6)
grid_search.fit(x_train,y_train)
print('最佳分数',grid_search.best_score_) #best_score_最佳得分
print('最佳参数',grid_search.best_params_) #best_params_最佳参数
predictions=grid_search.predict(x_test)
三、引用sklearn.svm库中的SVC函数进行图片分类
from sklearn.pipeline import Pipeline
from sklearn.preprocessing import scale
from sklearn.model_selection import train_test_split,GridSearchCV
from sklearn.svm import SVC
from sklearn.metrics import classification_report
if __name__ == '__main__':
datas=np.array(datas)
datas_lables=np.array(datas_lables)
x_train,x_test,y_train,y_test = train_test_split(datas,datas_lables,test_size=0.05,random_state=1)
pipeline=Pipeline([('clf',SVC(kernel='rbf',gamma=0.01,C=100))])
parameters={'clf__gamma':(0.005,0.01,0.03,0.1,0.3,1),
'clf__C':(0.1,0.3,1,3,10,30)}
grid_search = GridSearchCV(pipeline,parameters,n_jobs=2,verbose=1,scoring='accuracy',cv=6)
grid_search.fit(x_train,y_train)
print('最佳分数',grid_search.best_score_)
print('最佳参数',grid_search.best_params_)
predictions=grid_search.predict(x_test)
print('预测得分',classification_report(y_test,predictions))
for i,j in zip(y_test,predictions):
print(i,j)