python爬虫下载重试_python爬虫多次请求超时的几种重试方法(6种)

第一种方法

headers = Dict()

url = 'https://www.baidu.com'

try:

proxies = None

response = requests.get(url, headers=headers, verify=False, proxies=None, timeout=3)

except:

# logdebug('requests failed one time')

try:

proxies = None

response = requests.get(url, headers=headers, verify=False, proxies=None, timeout=3)

except:

# logdebug('requests failed two time')

print('requests failed two time')

总结 ：代码比较冗余，重试try的次数越多，代码行数越多，但是打印日志比较方便

第二种方法

def requestDemo(url，):

headers = Dict()

trytimes = 3 # 重试的次数

for i in range(trytimes):

try:

proxies = None

response = requests.get(url, headers=headers, verify=False, proxies=None, timeout=3)

# 注意此处也可能是302等状态码

if response.status_code == 200:

break

except:

# logdebug(f'requests failed {i}time')

print(f'requests failed {i} time')

总结 ：遍历代码明显比第一个简化了很多，打印日志也方便

第三种方法

def requestDemo(url， times=1):

headers = Dict()