求大佬指点,写的是三国演义中文文本的词频统计,可是总是报错(已解决)

1.首先是这样写的

import jieba
txt = open("D:/python程序/threekingdoms.txt","rt",encoding='utf-8').read()
words=jieba.lcut(txt)
counts={}
for word in words:
    if len(word)==1:
        continue
else:
    counts[word]=counts.get(word,0)+1
items=list(counts.items())
items.sort(key=lambda x:x[1],reverse=True)
for i in range(15):
    word,count=items[i]
    print("{0:<10}{1:<5}".format(word,count))

运行报错

Traceback (most recent call last):
  File "D:/python程序/三国词频统计.py", line 2, in <module>
    txt = open("D:/python程序/threekingdoms.txt","rt",encoding='utf-8').read()
  File "D:\PYthon37\lib\codecs.py", line 322, in decode
    (result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb5 in position 0: invalid start byte

2.后来改了一下,把’utf-8’改成了’unicode_escape’,还是报错

Building prefix dict from the default dictionary ...
Loading model from cache C:\Users\asus\AppData\Local\Temp\jieba.cache
Loading model cost 0.635 seconds.
Prefix dict has been built successfully.

Traceback (most recent call last):
         1    
  File "D:/python程序/三国词频统计.py", line 13, in <module>
    word,count=items[i]
IndexError: list index out of range

3.上网上搜又改,把’unicode_escape’改成’gbk’,仍然报错:

Traceback (most recent call last):
  File "D:/python程序/三国词频统计.py", line 2, in <module>
    txt = open("D:/python程序/threekingdoms.txt","rt",encoding='gbk').read()
UnicodeDecodeError: 'gbk' codec can't decode byte 0xfd in position 351251: illegal multibyte sequence

救救孩子吧,真的不知道怎么改了,求高手指点。

更新:已解决
其实改那些都不对,只要在创建txt文件保存的时候将格式编码选为“UTF-8”就可以了。额额,默认是ANSI。

之前一直都是它的默认选项ANSI
这样还是那段代码

import jieba
text = open('D:/三国演义.txt','r',encoding='utf-8').read()
words = jieba.lcut(text)
counts = {}
for word in words:
    if len(word) == 1:
        continue
    else:
        counts[word] = counts.get(word,0) + 1
items = list(counts.items())
items.sort(key = lambda x:x[1],reverse=True)
for i in range(10):
    word,count = items[i]
    print("{0:<10}{1:>5}".format(word,count))

运行结果:
在这里插入图片描述
最终版本为:

import jieba
text = open('D:/三国演义.txt','r',encoding='utf-8').read()
excludes={"将军","却说","荆州","二人","不可","不能","如此","丞相","商议","如何","主公","军士",
          "左右","军马","引兵","次日","大喜","天下","东吴","于是","今日","不敢","魏兵","陛下",
          "一人","都督","人马","不知"}
words = jieba.lcut(text)
counts = {}
for word in words:
    if len(word) == 1:
        continue
    elif word == "诸葛亮" or word =="孔明曰":
        rword="孔明"
    elif word == "关公" or word =="云长":
        rword="关羽"
    elif word == "玄德" or word =="玄德曰":
        rword="刘备"
    elif word == "孟德" or word =="丞相曰":
        rword="曹操"
    else:
        rword=word
    counts[rword] = counts.get(rword,0) + 1
for word in excludes:
    del counts[word]
items = list(counts.items())
items.sort(key = lambda x:x[1],reverse=True)
for i in range(10):
    word,count = items[i]
    print("{0:<10}{1:>5}".format(word,count))

运行结果:
在这里插入图片描述

评论 2
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值