import urllib.request
import urllib.parse
url ='https://www.xxx.com/s?wd='
headers ={
'User-Agent':'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36'
}
#变成unicode方式
# urllib.parse方法
request = urllib.request.Request(url = url,headers = headers)
name = urllib.parse.quote('毛不易')
url = url +name
print(url)
response = urllib.request.urlopen(request)
print(response.read().decode('utf-8'))
上面的例子中我们使用到了urllib.parse
主要的方法是urllib.parse.quote('xxx')将xxx中文信息转化为unicode编码,否则的话无法爬取到信息