python 异步_异步python容易出错

本文探讨了在Python异步Web服务器(如Sanic)中使用非异步库可能导致的问题,如应用程序响应慢甚至崩溃。通过示例展示了如何通过`run_in_executor`避免阻塞主进程,以及在编写异步代码时应优先考虑使用异步库以获得最佳性能。
摘要由CSDN通过智能技术生成

python 异步

It’s easy to write asynchronous Python code that does not behave like you think it would. I recently faced a situation where a Sanic web application was crashing/restarting in Kubernetes very often, but without any trace.

编写行为不像您想象的那样的异步Python代码很容易。 最近,我遇到了Sanic Web应用程序经常在Kubernetes中崩溃/重新启动的情况,但是没有任何痕迹。

Why was it crashing/restarting without any logs? I was clueless. The only possible cause that came to my mind was the readiness/liveness probe that I configured on that application. Then, it would have meant that application was not able to answer health checks — at least not fast enough.

为什么在没有任何日志的情况下崩溃/重新启动? 我一无所知。 我想到的唯一可能原因是我在该应用程序上配置的“就绪/活跃”探针。 然后,这将意味着应用程序无法回答运行状况检查-至少不够快。

It could also be the main process that was stuck doing something, and not able to serve new requests.

这也可能是停留在执行某项操作且无法满足新请求的主要过程。

Happens that was a right call, and both statements are true.

发生的是正确的选择,两个陈述都是正确的。

As surprising as it sounds, the following code could block your whole process during library method processing time, if the library you use is not built to support AsyncIO :

由于令人惊讶,因为它的声音,下面的代码可以在阻止你的整个过程 库方法的处理时间(如果您使用的库构建为支持AsyncIO):

@app.route("/my-api")
async def my_api(request):
res = some_method()
return json(res)


def block_method():
import somelib
somelib.call_some_method()

Considering the number of libraries we use, it is an easy mistake to do. However, it is possible to use blocking libraries in async application server without too much performance penalties.Let’s now investigate different implementations.

考虑到我们使用的库的数量,这是一个容易犯的错误。 但是,可以在异步应用程序服务器中使用阻塞库而不会造成过多的性能损失。现在让我们研究不同的实现。

不惜一切代价避免的事情 (What you should avoid at all cost)

@app.route("/status")
async def status(request):
return json("ok")@app.route("/sync")
async def to_avoid(request):
block_method()
return json({"hello": "i am slow and blocking"})

def block_method():
time.sleep(5)

We have 2 APIs here:

我们在这里有2个API:

  • one that performs basic health checks

    进行基本健康检查的人
  • the other that contains a call to a third party library

    另一个包含对第三方库的调用

If the third party library method does not contain “async” in his signature then it is not “async ready”. Let’s see what is going on when we call this API 5 times while simulating a 5 seconds processing time, and our health check endpoint just after that :

如果第三方库方法的签名中不包含“异步” ,则说明它不是“异步就绪”。 让我们看看在模拟5秒的处理时间时,调用此API 5次并发生了什么,以及此后的运行状况检查端点发生了什么:

Image for post

Wow, our health check takes more than 25 seconds to answer. This is a dangerous solution! Our health check API, like any other API we define, won’t be able to process any requests while the main process is stuck doing synchronous work. Consider deploying this solution over Kubernetes: it will tag the pod as unhealthy and restart it over and over again without throwing any error in the logs the moment you have requests coming in!

哇,我们的健康检查需要25秒钟以上才能回答。 这是一个危险的解决方案! 与我们定义的任何其他API一样,我们的运行状况检查API在主进程停留在进行同步工作时将无法处理任何请求。 考虑在Kubernetes上部署此解决方案:它将吊舱标记为不正常,并一遍又一遍地重新启动它,而在收到请求时不会在日志中引发任何错误!

你应该喜欢什么 (What you should prefer)

Good news : you actually can call a synchronous method from asynchronous without blocking the main process using run_in_executor :

好消息:实际上,您可以使用run_in_executor从异步调用同步方法而不会阻塞主进程:

@app.route("/async-sync")
async def prefer(request):
await loop.run_in_executor(None, block_method)
return json({"hello": "blocking io launched asynchronously"})def block_method():
time.sleep(5)

Let’s see what is going on when we call this API 5 times while simulating a 5 seconds processing time, and our health check endpoint just after that :

让我们看看在模拟5秒的处理时间时,调用此API 5次并发生了什么,以及此后的运行状况检查端点发生了什么:

Image for post

The main problem solved is now solved: our health check is not waiting to answer! On the other hand, the 5 previous calls are processed synchronously and taking 25 seconds to execute in total. So it is not an optimal solution but is not a show stopper either

解决的主要问题是现在解决了:我们的健康检查等待答案! 另一方面,之前的5个调用被同步处理,总共花费25秒来执行。 所以这不是一个最佳的解决方案,但也不是一个秀场停止者

如何获得最佳性能(What to do to get optimal performances)

Just use async libraries !

只需使用异步库!

@app.route("/async")
async def fully_async(request):
await asyncio.sleep(5)
return json({"hello": "async"})
Image for post

As we can see, in an ideal world where all our libraries are asynchronous, requests are processed fast : 5 requests in 5 seconds in total!

如我们所见,在所有库都是异步的理想世界中,请求被快速处理:总共5秒内有5个请求!

We witnessed how easy it is to write sub-optimal code for an asynchronous Python web server like Sanic. In a real-world scenario, we also need to use libraries to implement fast and avoid re-inventing the wheel. Next time you add a library to your requirements, be extra cautious about the libraries you take and prefer those who offer asynchronous support. If no alternative exists then, instead of rewriting the whole library, use loop.run_in_executor to execute synchronous calls without blocking the main process.

我们见证了为Sanic这样的异步Python Web服务器编写次优代码是多么容易。 在现实世界中,我们还需要使用库来快速实现并避免重新发明轮子。 下次在您的需求中添加库时,请特别注意要使用的库,并优先选择提供异步支持的库。 如果不存在替代方法,则可以使用loop.run_in_executor来执行同步调用,而不用阻塞整个进程,而不必重写整个库。

翻译自: https://medium.com/@victorfleurant/async-python-its-easy-to-do-it-wrong-9cfacb9a946e

python 异步

  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值