文件方式实现完整的英文词频统计实例

文件方式实现完整的英文词频统计实例

可以下载一长篇的英文小说，进行词频的分析。

1.读入待分析的字符串

2.分解提取单词 

3.计数字典

4.排除语法型词汇

5.排序

6.输出TOP(20)

7.对输出结果的简要说明。

str='''I've been reading books of old
The legends and the myths
Achilles and his gold
Hercules and his gifts
Spiderman's control
And Batman with his fists
And clearly I don't see myself upon that list
She said, where'd you wanna go?
How much you wanna risk?
I'm not looking for somebody
With some superhuman gifts
Some superhero
Some fairytale bliss
Just something I can turn to
Somebody I can kiss
I want something just like this
Doo-doo-doo, doo-doo-doo
Doo-doo-doo, doo-doo
Doo-doo-doo, doo-doo-doo
Oh I want something just like this
Doo-doo-doo, doo-doo-doo
Doo-doo-doo, doo-doo
Doo-doo-doo, doo-doo-doo
Oh I want something just like this
I want something just like this
I've been reading books of old
The legends and the myths
The testaments they told
The moon and its eclipse
And Superman unrolls
A suit before he lifts
But I'm not the kind of person that it fits
She said, where'd you wanna go?
How much you wanna risk?
I'm not looking for somebody
With some superhuman gifts
Some superhero
Some fairytale bliss
Just something I can turn to
Somebody I can miss
I want something just like this
I want something just like this
Oh I want something just like this
Doo-doo-doo, doo-doo-doo
Doo-doo-doo, doo-doo
Doo-doo-doo, doo-doo-doo
Oh I want something just like this
Doo-doo-doo, doo-doo-doo
Doo-doo-doo, doo-doo
Doo-doo-doo, doo-doo-doo
Where'd ya wanna go?
How much you wanna risk?
I'm not looking for somebody
With some superhuman gifts
Some superhero
Some fairytale bliss
Just something I can turn to
Somebody I can kiss
I want something just like this
Oh I want something just like this
Oh I want something just like this
Oh I want something just like this'''

fo=open('test.txt','w')
fo.write('''I've been reading books of old
The legends and the myths
Achilles and his gold
Hercules and his gifts
Spiderman's control
And Batman with his fists
And clearly I don't see myself upon that list
She said, where'd you wanna go?
How much you wanna risk?
I'm not looking for somebody
With some superhuman gifts
Some superhero
Some fairytale bliss
Just something I can turn to
Somebody I can kiss
I want something just like this
Doo-doo-doo, doo-doo-doo
Doo-doo-doo, doo-doo
Doo-doo-doo, doo-doo-doo
Oh I want something just like this
Doo-doo-doo, doo-doo-doo
Doo-doo-doo, doo-doo
Doo-doo-doo, doo-doo-doo
Oh I want something just like this
I want something just like this
I've been reading books of old
The legends and the myths
The testaments they told
The moon and its eclipse
And Superman unrolls
A suit before he lifts
But I'm not the kind of person that it fits
She said, where'd you wanna go?
How much you wanna risk?
I'm not looking for somebody
With some superhuman gifts
Some superhero
Some fairytale bliss
Just something I can turn to
Somebody I can miss
I want something just like this
I want something just like this
Oh I want something just like this
Doo-doo-doo, doo-doo-doo
Doo-doo-doo, doo-doo
Doo-doo-doo, doo-doo-doo
Oh I want something just like this
Doo-doo-doo, doo-doo-doo
Doo-doo-doo, doo-doo
Doo-doo-doo, doo-doo-doo
Where'd ya wanna go?
How much you wanna risk?
I'm not looking for somebody
With some superhuman gifts
Some superhero
Some fairytale bliss
Just something I can turn to
Somebody I can kiss
I want something just like this
Oh I want something just like this
Oh I want something just like this
Oh I want something just like this''')
fo.close()

fo=open('text.txt','r')
str=fo.read()
fo.close()

#字母转换小写
str = str.lower()

#转换标点符号为空格
for i in ",.?":
    str =str.replace(i," ")

#每一个单词的分隔
words=str.split(' ')


#字典转换集合并计数单词出现次数
dic={}
keys=set(words)
for w in keys:
    dic[w]=words.count(w)


#单词计数元组的列表
wc=list(dic.items())

#排序
wc.sort(key=lambda x:x[1], reverse=True)


for i in range(20):
    print(wc[i])

posted on 2017-09-27 16:03 068冯斐然 阅读( ...) 评论( ...) 编辑 收藏

转载于:https://www.cnblogs.com/ffr0068/p/7602248.html