本想做个爬虫试一下,结果从老文档里学来的代码一直报错,URLError没有code成员,出于对老文献的信任,蒙蔽了半天。最后灵机一动选中URLError右键查看定义,才发现code被挪到HTTPError里面去了,终于真相大白,把URLError换成HTTPError就好了。
看来作为一个小白,以后还得提高勤于看库文件定义的姿势水平啊。
def download(url_,user_agent = 'asp',num_retries = 5):
print("Downloading:",url_)
headers = {'User-agent':user_agent}
req = urllib.request.Request(url_, headers = headers)
try:
html = urllib.request.urlopen(req).read()
except urllib.request.URLError as e:
print("DownloadError!",e.reason)
html = None
if num_retries > 0:
if hasattr(e,'code') and 500 <= e.code <= 600:
return download(url_,num_retries - 1)
return html
把URLError换成HTTPError
except urllib.request.HTTPError as e: