1.首先是这样写的:
import jieba
txt = open("D:/python程序/threekingdoms.txt","rt",encoding='utf-8').read()
words=jieba.lcut(txt)
counts={}
for word in words:
if len(word)==1:
continue
else:
counts[word]=counts.get(word,0)+1
items=list(counts.items())
items.sort(key=lambda x:x[1],reverse=True)
for i in range(15):
word,count=items[i]
print("{0:<10}{1:<5}".format(word,count))
运行报错:
Traceback (most recent call last):
File "D:/python程序/三国词频统计.py", line 2, in <module>
txt = open("D:/python程序/threekingdoms.txt","rt",encoding='utf-8').read()
File "D:\PYthon37\lib\codecs.py", line 322, in decode
(result, consumed) = self._buffer_decode(data, self.errors, final)
UnicodeDecodeError: 'utf-8' codec can't decode byte 0xb5 in position 0: invalid start byte
2.后来改了一下,把’utf-8’改成了’unicode_escape’,还是报错:
Building prefix dict from the default dictionary ...
Loading model from cache C:\Users\asus\AppData\Local\Temp\jieba.cache
Loading model cost 0.635 seconds.
Prefix dict has been built successfully.
Traceback (most recent call last):
1
File "D:/python程序/三国词频统计.py", line 13, in <module>
word,count=items[i]
IndexError: list index out of range
3.上网上搜又改,把’unicode_escape’改成’gbk’,仍然报错:
Traceback (most recent call last):
File "D:/python程序/三国词频统计.py", line 2, in <module>
txt = open("D:/python程序/threekingdoms.txt","rt",encoding='gbk').read()
UnicodeDecodeError: 'gbk' codec can't decode byte 0xfd in position 351251: illegal multibyte sequence
救救孩子吧,真的不知道怎么改了,求高手指点。
更新:已解决
其实改那些都不对,只要在创建txt文件保存的时候将格式编码选为“UTF-8”就可以了。额额,默认是ANSI。
这样还是那段代码:
import jieba
text = open('D:/三国演义.txt','r',encoding='utf-8').read()
words = jieba.lcut(text)
counts = {}
for word in words:
if len(word) == 1:
continue
else:
counts[word] = counts.get(word,0) + 1
items = list(counts.items())
items.sort(key = lambda x:x[1],reverse=True)
for i in range(10):
word,count = items[i]
print("{0:<10}{1:>5}".format(word,count))
运行结果:
最终版本为:
import jieba
text = open('D:/三国演义.txt','r',encoding='utf-8').read()
excludes={"将军","却说","荆州","二人","不可","不能","如此","丞相","商议","如何","主公","军士",
"左右","军马","引兵","次日","大喜","天下","东吴","于是","今日","不敢","魏兵","陛下",
"一人","都督","人马","不知"}
words = jieba.lcut(text)
counts = {}
for word in words:
if len(word) == 1:
continue
elif word == "诸葛亮" or word =="孔明曰":
rword="孔明"
elif word == "关公" or word =="云长":
rword="关羽"
elif word == "玄德" or word =="玄德曰":
rword="刘备"
elif word == "孟德" or word =="丞相曰":
rword="曹操"
else:
rword=word
counts[rword] = counts.get(rword,0) + 1
for word in excludes:
del counts[word]
items = list(counts.items())
items.sort(key = lambda x:x[1],reverse=True)
for i in range(10):
word,count = items[i]
print("{0:<10}{1:>5}".format(word,count))
运行结果: