python爬虫超时重试_python爬虫URL重试机制实现(python2.7以及python3.5)

应用场景：

状态不是200的URL重试多次

代码比较简单还有部分注释

python2.7实现：

# -*-coding:utf-8-*-

"""

ayou

"""

import requests

def url_retry(url,num_retries=3):

print("access!")

try:

request = requests.get(url,timeout=60)

#raise_for_status(),如果不是200会抛出HTTPError错误

request.raise_for_status()

html = request.content

except requests.HTTPError as e:

html=None

if num_retries>0:

#如果不是200就重试，每次递减重试次数

return url_retry(url,num_retries-1)

#如果url不存在会抛出ConnectionError错误，这个情况不做重试

except requests.exceptions.ConnectionError as e:

return

return html

url_retry("http://httpbin.org/status/404")

python3.5实现：

# -*-coding:utf-8-*-

"""

ayou