requests库结合asyncio
-
使用asyncio和requests库
-
由于request是同步的,会阻塞asyncio
-
所以为每个request请求都创建多一个事件轮询器(这边我理解是多一个事件轮询器对应一个多线程)去跑requests库的请求
-
# 使用requests库 def get_resp(xx): resp = requests.get(xx) return resp # 每个request请求都创建多一个事件轮询器 async def async_requests(): loop = asyncio.get_event_loop() resp = await loop.run_in_executor(None,get_resp,xx) # 将任务添加到执行队列中去 async def tasks(): tasks = [async_requests() for i in range(10)] ret = await asyncio.gather(*tasks) return ret # 主入口 def main(): asyncio.run(tasks()) if '__name__' == '__main__': main()
参考文章:
- https://blog.csdn.net/J_Object/article/details/78880610
- https://stackoverflow.com/questions/53020839/asyncio-post-requests-with-login-to-site/53028466