【无标题】

 

 接下来就是可视化操作了：

       1，利用jieba 分词

Python的中文分词库有很多，常见的有：jieba,thulac,pkuseg,snownlp等​​​​​​

“结巴”中文分词：做最好的 Python 中文分词组件

• 支持三种分词模式：

  • 精确模式，试图将句子最精确地切开，适合文本分析；
  • 全模式，把句子中所有的可以成词的词语都扫描出来, 速度非常快，但是不能解决歧义；
  • 搜索引擎模式，在精确模式的基础上，对长词再次切分，提高召回率，适合用于搜索引擎分词。

• 支持繁体分词

• 支持自定义词典

 

from collections import Counter
import jieba
import pandas as pd
from pprint import pprint

df = pd.read_csv("take off.csv",engine="python",encoding="utf-8-sig",header=None)
df.index = range(len(df))
df.columns = ["内容","uname","sex"]
df['cut'] = df['内容'].apply(lambda x:list(set(jieba.cut(x))))

print(df['cut'].loc[0])#输出第一条信息的分词情况

2.网上下载停用词表，利用停用词表对分词进行过滤

# 读取停用词数据
stopwords = pd.read_csv('hit_stopwords.txt',encoding='utf-8',engine='python',names=['stopword'],header=None,sep='None')
print(df['cut'].loc[1600])
stopwords.head()
stop_list = stopwords['stopword'].to_list()
# 去除停用词
df['cut'] = df['内容'].apply(lambda x : [i for i in jieba.cut(x) if i not in stop_list])
print(df['cut'].loc[1600])
df.head()

3.利用collection 中的counter 统计词频并存入csv文件

words=[]
for content in df['cut']:
    words.extend(content)
counter = Counter(words)
print(counter.most_common(100))
name=['词','频']
df1=pd.DataFrame(columns=name,data=counter.most_common(100))
df1.to_csv("third_statistics.csv",header=None,index=None,encoding="utf-8-sig")

 

 

 数据准备好了就开始pyecharts可视化操作了

1.柱状图

 2.圆饼图

from pyecharts import options as opts
from pyecharts.charts import Page, Pie
import pandas as pd
df = pd.read_csv("third_statistics.csv",engine="python",encoding="utf-8-sig")
df.columns=['词','频']
wd = df['词'][0:50]
num = df['频'][0:50]
c = Pie()
c.add("",[list(z) for z in zip(wd, num)],radius = ["40%", "75%"],center = ['70%','50%'])
# 圆环的粗细和大小
c.set_global_opts( title_opts=opts.TitleOpts(title="Pie-Radius"),legend_opts=opts.LegendOpts( orient="vertical", pos_top="5%", pos_left="2%" ))
c .set_series_opts(label_opts=opts.LabelOpts(formatter="{b}: {c}"))
c.render_notebook()
c.render("pie.html")

 

4.词条云

from pyecharts import options as opts
from pyecharts.charts import WordCloud
import pandas as pd

df = pd.read_csv("third_statistics.csv",engine="python",encoding="utf-8-sig")
df.index = range(len(df))
df.columns = ["词","频"]
data=[]
for i in df.index:
    data.append(tuple(df.values[i]))
print(data)
wordcloud = (
    WordCloud()
    .add("", data_pair = data, word_size_range=[20,100],textstyle_opts=opts.TextStyleOpts(font_family='Microsoft YaHei',font_weight='bold'))
    .set_global_opts(title_opts=opts.TitleOpts(title="b站词条云",title_textstyle_opts = opts.TextStyleOpts(font_size = 25,color="midnightblue")))
    .render("wordcloud.html")
)

另外可视化工具的使用方法还需讲解的后面再补充 

码字不易，球球点个关注，感谢感谢！：）