如题,最近在使用requests包进行一些RESTFul api的编程,有一部分涉及到retry的操作,requests包提供了非常方便的配置retry的方法,具体的网上教程很多,这里不做赘述,直接贴一下我自己的代码。
def create_session() -> requests.Session:
retry_strategy = Retry(
total=TeeControlClient._POST_RETRY_COUNT,
status_forcelist=[429, 500, 502, 503, 504],
method_whitelist=["POST"]
)
adapter = requests.adapters.HTTPAdapter(max_retries=retry_strategy)
session = requests.Session()
session.mount("https://", adapter)
session.mount("http://", adapter)
return session
我这个是专门给POST方法配置的,所以method_whitelist配置的只有一个POST。
这个配置看起来没啥问题,然后大家都知道requests的api都应该配置一个timeout,我有一个测试用例就是连接一个url,因为proxy的原因会卡住不动。具体代码如下
session = create_session()
try:
return session.post(url, json=json_data, timeout=10)
except requests.exceptions.RequestException as error:
logger.error("Cannot connect to %s, Reason: %s", url, error)
然后执行代码发现直接卡死,timeout没有起作用!
多方查证之后发现是因为这种因为我的case是因为网络原因,使用了Adapter之后,需要配置Retry的connect属性
connect (int) –
How many connection-related errors to retry on.
These are errors raised before the request is sent to the remote server, which we assume has not triggered the server to process the request.
Set to
0
to fail on the first retry of this type.
直接代码改成:
def create_session() -> requests.Session:
""" create a Session with Retry Adapter
Returns:
Session with Retry Adapter
"""
retry_strategy = Retry(
total=TeeControlClient._POST_RETRY_COUNT,
status_forcelist=[429, 500, 502, 503, 504],
method_whitelist=["POST"],
connect=0,
)
adapter = requests.adapters.HTTPAdapter(max_retries=retry_strategy)
session = requests.Session()
session.mount("https://", adapter)
session.mount("http://", adapter)
return session
问题搞定,retry和timeout都生效了
其他还有 read 等属性,可以参考一下 urllib3 ,里面搜索一下retry就能找到了