DC竞赛达观杯文本分类竞赛简单逻辑回归解答

运用sklearn库将词向量化，然后直接逻辑回归解决分类问题

导入模块

import pandas as pd
from sklearn.linear_model import LogisticRegression
from sklearn.feature_extraction.text import CountVectorizer

读取数据，去掉多余数据

df_train = pd.read_csv('./train_set.csv')
df_test = pd.read_csv('./test_set.csv')
df_train.drop(columns=['article','id'],inplace = True)
df_test.drop(columns=['article'],inplace=True)

查看数据头

df_train.head()
df_test.head()

向量化词

vectorizer = CountVectorizer(ngram_range=(1,2),min_df=3,max_df=0.9,
                             max_features=100000)
vectorizer.fit(df_train['word_seg'])
x_train = vectorizer.transform(df_train['word_seg'])
x_test = vectorizer.transform(df_test['word_seg'])
y_train = df_train['class']-1

查看向量化的输入向量

x_train.shape
x_test.shape
y_train.shape

逻辑回归

lg = LogisticRegression(C=4,dual=True)
lg.fit(x_train,y_train)
y_test = lg.predict(x_test)

输出预测数据

df_test['class'] = y_test.tolist()
df_test['class']=df_test['class']+1
df_result = df_test.loc[:,['id','class']]
df_result.to_csv('./result.csv',index=False)