Python 文本词频统计

最新推荐文章于 2025-02-15 21:15:30 发布

cycy小陈

最新推荐文章于 2025-02-15 21:15:30 发布

阅读量2.2w

点赞数 10

分类专栏： # Python程序设计

本文链接：https://blog.csdn.net/qq_40390825/article/details/82387991

版权

Python程序设计专栏收录该内容

19 篇文章

订阅专栏

本文介绍了使用Python进行经典文学作品《哈姆雷特》及《三国演义》的词频统计方法。展示了如何清洗文本数据、提取单词并统计频率，最终输出了两部作品中出现频率最高的词汇。

摘要生成于 C知道，由 DeepSeek-R1 满血版支持，前往体验 >

Hamlet文本词频统计


The Tragedy of Hamlet, Prince of Denmark
Shakespeare homepage | Hamlet | Entire play
ACT I

SCENE I. Elsinore. A platform before the castle.

FRANCISCO at his post. Enter to him BERNARDO
BERNARDO
Who's there?
FRANCISCO
Nay, answer me: stand, and unfold yourself.
BERNARDO
Long live the king!
FRANCISCO
Bernardo?
BERNARDO
He.
FRANCISCO
You come most carefully upon your hour.
...
...
...

代码：

def getText():
    txt = open("hamlet.txt", "r").read()
    txt = txt.lower()
    for ch in '!"#$%&()*+,-./;:<=>?@[\\]^‘_{|}~':
        txt = txt.replace(ch, " ")
    return txt

hamletTxt = getText()
words = hamletTxt.split()
counts = {}
for word in words:
    counts[word] = counts.get(word, 0) + 1
items = list(counts.items())
items.sort(key=lambda x:x[1], reverse=True)
for i in range(10):
    word, count = items[i]
    print("{0:<10}{1:>5}".format(word, count))

输出频次最高的前十个单词：

the        1138
and         965
to          754
of          669
you         550
i           542
a           542
my          514
hamlet      462
in          436

输出三国演义出现次数最多的十个词汇

代码：

# -*- coding: UTF-8 -*-
import jieba
txt = open("threekingdoms.txt", "r", encoding="utf-8").read()
words = jieba.lcut(txt)
counts = {}
for word in words:
    if len(word) == 1:
        continue
    else:
        counts[word] = counts.get(word, 0) + 1
items = list(counts.items())
items.sort(key=lambda x:x[1], reverse=True)
for i in range(15):
    word, count = items[i]
    print("{0:<10}{1:<5}".format(word, count))

输出：

曹操        953  
孔明        836  
将军        772  
却说        656  
玄德        585  
关公        510  
丞相        491  
二人        469  
不可        440  
荆州        425  
玄德曰       390  
孔明曰       390  
不能        384  
如此        378  
张飞        358

通过测试，统计三国演义出现次数最多的前十个人物名称

代码：

# -*- coding: UTF-8 -*-
import jieba
txt = open("threekingdoms.txt", "r", encoding="utf-8").read()
excludes = {"将军", "却说", "荆州", "二人", "不可", "不能", "如此", "主公",\
            "军士", "商议", "如何", "左右", "军马", "引兵", "次日", "大喜",\
            "天下", "东吴", "于是", "今日", "不敢", "魏兵", "陛下", "一人",\
            "都督", "人马", "不知"}
words = jieba.lcut(txt)
counts = {}
for word in words:
    if len(word) == 1:
        continue
    elif word == "诸葛亮" or word == "孔明曰":
        rword = "孔明"
    elif word == "关公" or word == "云长":
        rword = "关羽"
    elif word == "玄德" or word == "玄德曰":
        rword = "刘备"
    elif word == "孟德" or word == "丞相":
        rword = "曹操"
    else:
        rword = word
    counts[rword] = counts.get(rword, 0) + 1
for word in excludes:
    del counts[word]
items = list(counts.items())
items.sort(key=lambda x:x[1], reverse=True)
for i in range(10):
    word, count = items[i]
    print("{0:<10}{1:<5}".format(word, count))

输出：

曹操        1451 
孔明        1383 
刘备        1252 
关羽        784  
张飞        358  
吕布        300  
赵云        278  
孙权        264  
司马懿       221  
周瑜        217