多文件分词并统计词频

最新推荐文章于 2023-06-23 08:35:07 发布

又要起名字了

最新推荐文章于 2023-06-23 08:35:07 发布

阅读量2.4k

点赞数 4

分类专栏：日常练习 jieba 文章标签： jieba 多文件分词词频统计

本文链接：https://blog.csdn.net/weixin_44530236/article/details/89387458

版权

该博客介绍了如何利用jieba库对多文件进行分词，并结合停词表去除停用词，统计每个文件的词频。通过读取文件内容，进行分词处理，然后构建词频字典，最后将结果输出到CSV文件中。

摘要由CSDN通过智能技术生成

import os
import jieba

数据集如下：

folder_path = r"C:\Users\Machine Learning\comments"
os.listdir(folder_path)

['201603枕.txt',
 '201603锅.txt',
 '201604枕.txt',
 '201604锅.txt',
 '201605枕.txt',
 '201605锅.txt',
 '201606枕.txt',
 '201606锅.txt',
 '201607枕.txt',
 '201607锅.txt',
 '201608枕.txt',
 '201608锅.txt',
 '201609枕.txt',
 '201609锅.txt',
 '201610枕.txt',
 '201610锅.txt',
 '201611枕.txt',
 '201611锅.txt',
 '201612枕.txt',
 '201612锅.txt',
 '201701枕.txt',
 '201701锅.txt',
 '201702枕.txt',
 '201702锅.txt',
 '201703枕.txt',
 '201703锅.txt',
 '201704枕.txt',
 '201704锅.txt',
 '201705枕.txt',
 '201705锅.txt',
 '201706枕.txt',
 '201706锅.txt',
 '201707枕.txt',
 '201707锅.txt',
 '201708枕.txt',
 '201708锅.txt',
 '201709枕.txt',
 '201709锅.txt',
 '201710枕.txt',
 '201710锅.txt',
 '201711枕.txt',
 '201711锅.txt',
 '201712枕.txt',
 '201712锅.txt',
 '201801枕.txt',
 '201801锅.txt',
 '201802枕.txt',
 '201802锅.txt',
 '201803锅.txt',
 '201804枕.txt',
 '201804锅.txt',
 '201805枕.txt',
 '201805锅.txt',
 '201806枕.txt',
 '201806锅.txt',
 '201807枕.txt',
 '201807锅.txt',
 '201808枕.txt',
 '201808锅.txt',
 '201809枕.txt',
 '201809锅.txt',
 '201810枕.txt',
 '201810锅.txt',
 '201811枕.txt',
 '201811锅.txt',
 '201812枕.txt',
 '201812锅.txt',
 '201901枕.txt',
 '201901锅.txt',
 '201902枕.tx