sklearn的pipeline的简单使用

基础模型:tfidf+Ridge
1.在不使用pipeline的情况下，模型是这样的:

# 模型定义与训练
tf = TfidfVectorizer(min_df= 3, max_df=0.5, analyzer = 'char_wb', ngram_range = (3,5))
tv_fit = tf.fit_transform(df['text'])
rf = Ridge()
rf.fit(tv_fit,df['y'])
# 模型测试
x1 = tf.transform(df_val['less_toxic'])
x2 = tf.transform(df_val['more_toxic'])
p1 = rf.predict(x1)
p2 = rf.predict(x2)
print(f'Validation Accuracy is { np.round((p1 < p2).mean() * 100,2)}')

输出:
Validation Accuracy is 68.45
2.在使用pipeline的情况下:

pipeline = Pipeline(
    [
        ("vect", TfidfVectorizer(min_df= 3, max_df=0.5, analyzer = 'char_wb', ngram_range = (3,5))),
        #("clf", RandomForestRegressor(n_estimators = 5, min_sample_leaf=3)),
        ("clf", Ridge()),
        #("clf",LinearRegression())
    ]
)
# Train the pipeline
pipeline.fit(df['text'], df['y'])
df_val = pd.read_csv("../input/jigsaw-toxic-severity-rating/validation_data.csv")
p1 = pipeline.predict(df_val['less_toxic'])
p2 = pipeline.predict(df_val['more_toxic'])
print(f'Validation Accuracy is { np.round((p1 < p2).mean() * 100,2)}')

输出:
Validation Accuracy is 68.45

推荐内容:https://blog.csdn.net/lanchunhui/article/details/50521648