基于贝叶斯算法之新闻分类

数据源：http://www.sogou.com/labs/resource/ca.php

数据源是一个搜狗新闻数据集，里面包含新闻的种类、主题、URL与内容，现在需要将新闻的内容进行清洗、去掉杂乱的字符串，并实现关键词提取、构造关键词云图，最后将整合后的内容进行分类。
首先导入数据：

import pandas as pd
import numpy as np
import jieba
import matplotlib.pyplot as plt

file_path = 'F:\\python_learing\\news_data\\val.txt'
def load_data(filename):
    df_news = pd.read_table(filename,sep='\t',names=['category','theme','URL','content'])
    df_news = df_news.dropna()
    return df_news

df_news = load_data(file_path)
df_news.head()

接下来列用jieba对内容进行分词：

content = df_news.content.values.tolist()

def jieba_cut(dataSet):
    content_S = []
    for line in dataSet:
        current = jieba.lcut(line)
        if len(current) > 1 and current != '\r\t':
            content_S.append(current)
    df_content = pd.DataFrame({'content_S':content_S})
    return df_content
df_content = jieba_cut(content)
df_content.head()