python统计段落单词词频_python词频统计实例

这篇文章主要介绍我最近刚学习的词频统计，权当作学习笔记，欢迎各位大佬批评指正。

hamlet英文词频统计：https://python123.io/resources/pye/hamlet.txt​python123.io

#CalHamletV1.py

def getText():

txt = open("hamlet.txt","r").read()

txt = txt.lower()

for ch in '!"#$%&()*+,-./:;<=>?@[\\]^_‘{|}~':

txt = txt.replace(ch," ")

return txt

hamletText = getText()

words = hamletText.split()

counts = {}

for word in words:

counts[word] = counts.get(word,0) + 1

items = list(counts.items())

items.sort(key = lambda x:x[1],reverse = True)

for i in range(10):

word,count = items[i]

print("{0:<10}{1:>5}".format(word,count))

其中，

def getText():

txt = open("hamlet.txt","r").read()

txt = txt.lower()

for ch in '!"#$%&()*+,-./:;<=>?@[\\]^_‘{|}~':

txt = txt.replace(ch," ")

return txt

主要是用来将文中的大小写全部变为小写，并将特殊字符用“ ”代替，将全文变为小写英文，以便全文的词频统计。

hamletText = getText()

words = hamletText.split()

counts = {}

for word in words:

counts[word] = counts.get(word,0) + 1

items = list(counts.items())

items.sort(key = lambda x:x[1],reverse = True)

这一块主要是将文本各个单词分开，并创建字典，遍历字典每一个单词，并利用counts.get()函数进行词频统计，并将字典键值对进行列表，利用items.sort()排序。

for i in range(10):

word,count = items[i]

print("{0:<10}{1:>5}".format(word,count))

将词频最高的十个单词及他们出现的次数显示出来。