Python段落计数令牌

最新推荐文章于 2024-09-14 08:25:16 发布

海澜明月

最新推荐文章于 2024-09-14 08:25:16 发布

阅读量397

点赞数

文章标签： python 开发语言

本文链接：https://blog.csdn.net/weixin_48077282/article/details/128380024

版权

本文演示了如何使用Python的nltk模块和split函数来统计文本中段落的单词数量。通过两个示例，展示了对于含有特殊字符如'(head)'的处理方式，分别计算出不同的单词计数。

摘要由CSDN通过智能技术生成

令牌有时也叫作标志，在从源读取文本时，有时我们需要找出有关所用单词的一些统计信息。这使得有必要计算单词的数量以及计算给定文本中具有特定类型单词的行数。在下面的示例中，我们展示了使用两种不同方法计算段落中单词的程序。假设这个示例文本中包含好莱坞电影的摘要。
读取文件
FileName = (“Path\GodFather.txt”)

with open(FileName, ‘r’) as file:
lines_in_file = file.read()
print lines_in_file
Python
当运行上面的程序时，得到以下输出 -
Vito Corleone is the aging don (head) of the Corleone Mafia Family. His youngest son Michael has returned from WWII just in time to see the wedding of Connie Corleone (Michael’s sister) to Carlo Rizzi. All of Michael’s family is involved with the Mafia, but Michael just wants to live a normal life. Drug dealer Virgil Sollozzo is looking for Mafia families to offer him protection in exchange for a profit of the drug money. He approaches Don Corleone about it, but, much against the advice of the Don’s lawyer Tom Hagen, the Don is morally against the use of drugs, and turns down the offer. This does not please Sollozzo, who has t