从网上抓取网站写下面这段代码时,发现报UnicodeEncodeError: 'gbk' codec can't encode character '\xXX' in position XX 错误
from urllib import request
req=request.Request("https://www.baidu.com")
req.add_header("User-Agent","Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:51.0) Gecko/20100101 Firefox/51.0")
resp=request.urlopen(req)
print(resp.read().decode('utf-8'))
查了一下发现了解决办法原来是print()函数自身有限制,不能完全打印所有的unicode字符。
其实print()函数的局限就是 Python默认编码的局限,因为系统是win7的,python的默认编码不是'utf-8',改一下python的默认编码成'utf-8'就行了import io
import sys
from urllib import request
sys.stdout = io.TextIOWrapper(sys.stdout.buffer,encoding='utf