Python开发【Tornado】：异步Web服务（二）

最新推荐文章于 2023-05-24 20:16:45 发布
最新推荐文章于 2023-05-24 20:16:45 发布
阅读量183 收藏
点赞数
文章标签： python
原文链接：http://www.cnblogs.com/lianzhilei/p/7821889.html
版权

真正的 Tornado 异步非阻塞

前言：

　　其中 Tornado 的定义是 Web 框架和异步网络库，其中他具备有异步非阻塞能力，能解决他两个框架请求阻塞的问题，在需要并发能力时候就应该使用 Tornado。

　　但是在实际使用过程中很容易把 Tornado 使用成异步阻塞框架，这样对比其他两大框架没有任何优势而言，本文就如何实现真正的异步非阻塞记录。

笔记：

  • 默认情况下tornado是单线程阻塞模式，如果阻塞所有请求都需要等待
  • tornado.web.asynchronous可以异步使用，得益于AsyncHTTPClient模块的配合使用，两者缺一不可
  • tornado.gen.coroutine严重依赖第三方库的使用，如果没有第三方库的支持则依然是阻塞模式
  • Tornado 提供了多种的异步编写形式：回调、Future、协程等，其中以协程模式最是简单和用的最多
  • Tornado 实现异步的多种方式：coroutine配合第三方库、启用多线程、使用celery等

 

1、使用 gen.coroutine 异步编程

在 Tornado 中两个装饰器：

  • tornado.web.asynchronous
  • tornado.gen.coroutine

 ① asynchronous 装饰器是让请求变成长连接的方式，必须手动调用 self.finish() 才会响应

class MainHandler(tornado.web.RequestHandler):
    @tornado.web.asynchronous
    def get(self):
        # bad 
        self.write("Hello, world")

asynchronous 装饰器不会自动调用self.finish() ，如果没有没有指定结束，该长连接会一直保持直到 pending 状态

所以正确是使用方式是使用了 asynchronous 需要手动 finish

class MainHandler(tornado.web.RequestHandler):
    @tornado.web.asynchronous
    def get(self):
        self.write("Hello, world")
        self.finish()

② coroutine 装饰器是指定改请求为协程模式，说明白点就是能使用 yield 配合 Tornado 编写异步程序。

Tronado 为协程实现了一套自己的协议，不能使用 Python 普通的生成器。

在使用协程模式编程之前要知道如何编写 Tornado 中的异步函数，Tornado 提供了多种的异步编写形式：回调、Future、协程等，其中以协程模式最是简单和用的最多。

编写一个基于协程的异步函数同样需要 coroutine 装饰器

@gen.coroutine
def sleep(self):
    yield gen.sleep(10)
    raise gen.Return([1, 2, 3, 4, 5])

 这就是一个异步函数，Tornado 的协程异步函数有两个特点：

  • 需要使用 coroutine 装饰器
  • 返回值需要使用 raise gen.Return() 当做异常抛出

返回值作为异常抛出是因为在 Python 3.2 之前生成器是不允许有返回值的。

使用过 Python 生成器应该知道，想要启动生成器的话必须手动执行 next() 方法才行，所以这里的 coroutine 装饰器的其中一个作用就是在调用这个异步函数时候自动执行生成器。

使用 coroutine 方式有个很明显是缺点就是严重依赖第三方库的实现，如果库本身不支持 Tornado 的异步操作再怎么使用协程也是白搭依然会是阻塞的，放个例子感受一下

import time
import logging
import tornado.ioloop
import tornado.web
import tornado.options
from tornado import gen

tornado.options.parse_command_line()

class MainHandler(tornado.web.RequestHandler):
    @tornado.web.asynchronous
    def get(self):
        self.write("Hello, world")
        self.finish()

class NoBlockingHnadler(tornado.web.RequestHandler):
    @gen.coroutine
    def get(self):
        yield gen.sleep(2)
        self.write('Blocking Request')


class BlockingHnadler(tornado.web.RequestHandler):
    @gen.coroutine
    def get(self):
        time.sleep(2)
        self.write('Blocking Request')

def make_app():
    return tornado.web.Application([
        (r"/", MainHandler),
        (r"/block", BlockingHnadler),
        (r"/noblock", NoBlockingHnadler),
    ], autoreload=True)

if __name__ == "__main__":
    app = make_app()
    app.listen(8001)
    tornado.ioloop.IOLoop.current().start()

当我们使用 yield gen.sleep(10) 这个异步的 sleep 时候，请求是不阻塞的。　　

[root@localhost ~]# siege http://192.168.0.2:8001/noblock -c100 -t10s
** SIEGE 3.0.8
** Preparing 100 concurrent users for battle.
The server is now under siege...
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.83 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.86 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.86 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.86 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.86 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.86 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.86 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.86 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.86 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.86 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.86 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.86 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.86 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.86 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.86 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.86 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.86 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.86 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.86 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.86 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.86 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.87 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.86 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.86 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.86 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.87 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.87 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.86 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.86 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.87 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.86 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.86 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.87 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.86 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.86 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.87 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.86 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.86 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.86 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.83 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.83 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.83 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.86 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.86 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.86 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.87 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.87 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.87 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.88 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.87 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.87 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.86 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.86 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.86 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.86 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.87 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.87 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.87 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.87 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.87 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.87 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.87 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.87 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.87 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.86 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.86 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.86 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.87 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.87 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.87 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.87 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.87 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.87 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.87 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.86 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.86 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.83 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.83 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.83 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.83 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.83 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.83 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.83 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.83 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.83 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.83 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.83 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.83 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.83 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.83 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.83 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.83 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.87 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.83 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.83 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.83 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.83 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.83 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.83 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.88 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.88 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.87 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.87 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.85 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.86 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.83 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.83 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.83 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.83 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.83 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.83 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.83 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.83 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.83 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.83 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.83 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.83 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.83 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.83 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.83 secs:      16 bytes ==> GET  /noblock
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /noblock

Lifting the server siege...      done.

Transactions:                 337 hits
Availability:              100.00 %
Elapsed time:                9.35 secs
Data transferred:            0.01 MB
Response time:                1.85 secs
Transaction rate:           36.04 trans/sec
Throughput:                0.00 MB/sec
Concurrency:               66.55
Successful transactions:         337
Failed transactions:               0
Longest transaction:            1.88
Shortest transaction:            1.83
 
FILE: /root/siege.log
You can disable this annoying message by editing
the .siegerc file in your home directory; change
the directive 'show-logfile' to false.
基准测试输出

测试结果：100并发，请求10s，共发送337次请求，平均每秒处理36.04次请求，很明显程序是非阻塞的

当使用 time.sleep(10) 时候请求时阻塞的。

[root@localhost ~]# siege http://192.168.0.2:8001/block -c100 -t10s
** SIEGE 3.0.8
** Preparing 100 concurrent users for battle.
The server is now under siege...
HTTP/1.1 200   1.84 secs:      16 bytes ==> GET  /block
HTTP/1.1 200   3.68 secs:      16 bytes ==> GET  /block
HTTP/1.1 200   5.51 secs:      16 bytes ==> GET  /block
HTTP/1.1 200   7.34 secs:      16 bytes ==> GET  /block
HTTP/1.1 200   9.17 secs:      16 bytes ==> GET  /block

Lifting the server siege...      done.

Transactions:                   5 hits
Availability:              100.00 %
Elapsed time:                9.23 secs
Data transferred:            0.00 MB
Response time:                5.51 secs
Transaction rate:            0.54 trans/sec
Throughput:                0.00 MB/sec
Concurrency:                2.98
Successful transactions:           5
Failed transactions:               0
Longest transaction:            9.17
Shortest transaction:            0.00
 
FILE: /root/siege.log
You can disable this annoying message by editing
the .siegerc file in your home directory; change
the directive 'show-logfile' to false.
基准测试输出

测试结果：100并发，请求10s，共发送5次请求，平均每秒处理0.5次请求，和time.sleep 2秒钟预先的效果是一致的。

gen.coroutine 在 Tornado 3.1 后会自动调用 self.finish() 结束请求，可以不使用 asynchronous 装饰器。

所以这种实现异步非阻塞的方式需要依赖大量的基于 Tornado 协议的异步库，使用上比较局限，好在还是有一些可以用的异步库

 

2、启用线程的处理异步编程

注：使用 gen.coroutine 装饰器编写异步函数，如果库本身不支持异步，那么响应任然是阻塞的。

 在 Tornado 中有个装饰器能使用 ThreadPoolExecutor 来让阻塞过程编程非阻塞，其原理是在 Tornado 本身这个线程之外另外启动一个线程来执行阻塞的程序，从而让 Tornado 变得非阻塞，根本原理就是启用多个线程处理，处理后的线程并不会自动关闭

import time
import logging
import tornado.ioloop
import tornado.web
import tornado.options
from tornado import gen
from tornado.concurrent import run_on_executor
from concurrent.futures import ThreadPoolExecutor

tornado.options.parse_command_line()

class MainHandler(tornado.web.RequestHandler):
    @tornado.web.asynchronous
    def get(self):
        self.write("Hello, world")
        self.finish()


class NoBlockingHnadler(tornado.web.RequestHandler):
    executor = ThreadPoolExecutor(4)              # 启动4个线程处理阻塞请求

    def sleep(self, second):
        time.sleep(second)
        return second

    @run_on_executor
    def get(self):
        second = yield self.sleep(5)
        self.write('noBlocking Request: {}'.format(second))

def make_app():
    return tornado.web.Application([
        (r"/", MainHandler),
        (r"/noblock", NoBlockingHnadler),
    ], autoreload=True)

if __name__ == "__main__":
    app = make_app()
    app.listen(8000)
    tornado.ioloop.IOLoop.current().start()

hreadPoolExecutor 是对标准库中的 threading 的高度封装，利用线程的方式让阻塞函数异步化，解决了很多库是不支持异步的问题。

但是与之而来的问题是，如果大量使用线程化的异步函数做一些高负载的活动，会导致该 Tornado 进程性能低下响应缓慢，这只是从一个问题到了另一个问题而已。

所以在处理一些小负载的工作，是能起到很好的效果，让 Tornado 异步非阻塞的跑起来。

但是明明知道这个函数中做的是高负载的工作，那么你应该采用另一种方式，使用 Tornado 结合 Celery 来实现异步非阻塞。

[root@localhost ~]# siege http://192.168.0.2:8001/noblock -c10 -t10s
** SIEGE 3.0.8
** Preparing 10 concurrent users for battle.
The server is now under siege...
HTTP/1.1 200   4.59 secs:      21 bytes ==> GET  /noblock
HTTP/1.1 200   4.59 secs:      21 bytes ==> GET  /noblock
HTTP/1.1 200   4.60 secs:      21 bytes ==> GET  /noblock
HTTP/1.1 200   4.60 secs:      21 bytes ==> GET  /noblock
HTTP/1.1 200   9.18 secs:      21 bytes ==> GET  /noblock
HTTP/1.1 200   8.18 secs:      21 bytes ==> GET  /noblock
HTTP/1.1 200   8.18 secs:      21 bytes ==> GET  /noblock
HTTP/1.1 200   8.18 secs:      21 bytes ==> GET  /noblock

Lifting the server siege...      done.

Transactions:                   8 hits
Availability:              100.00 %
Elapsed time:                9.42 secs
Data transferred:            0.00 MB
Response time:                6.51 secs
Transaction rate:            0.85 trans/sec
Throughput:                0.00 MB/sec
Concurrency:                5.53
Successful transactions:           8
Failed transactions:               0
Longest transaction:            9.18
Shortest transaction:            0.00
 
FILE: /root/siege.log
You can disable this annoying message by editing
the .siegerc file in your home directory; change
the directive 'show-logfile' to false.
基准测试输出

 

3、基于 Celery 的异步编程

Celery 是一个简单、灵活且可靠的，处理大量消息的分布式系统，专注于实时处理的任务队列，同时也支持任务调度。

Celery 并不是唯一选择，你可选择其他的任务队列来实现，但是 Celery 是 Python 所编写，能很快的上手，同时 Celery 提供了优雅的接口，易于与 Python Web 框架集成等特点。

与 Tornado 的配合可以使用 tornado-celery ，该包已经把 Celery 封装到 Tornado 中，可以直接使用（本节暂未测试，文章转载）

实际测试中，由于 tornado-celery 很久没有更新，导致请求会一直阻塞，不会返回

解决办法是：

把 celery 降级到 3.1 pip install celery==3.1
把 pika 降级到 0.9.14 pip install pika==0.9.14

Tornado服务端：

import time
import logging
import tornado.ioloop
import tornado.web
import tornado.options
from tornado import gen

import tcelery, tasks

tornado.options.parse_command_line()
tcelery.setup_nonblocking_producer()


class MainHandler(tornado.web.RequestHandler):
    @tornado.web.asynchronous
    def get(self):
        self.write("Hello, world")
        self.finish()


class CeleryHandler(tornado.web.RequestHandler):
    @gen.coroutine
    def get(self):
        response = yield gen.Task(tasks.sleep.apply_async, args=[5])
        self.write('CeleryBlocking Request: {}'.format(response.result))


def make_app(): 
    return tornado.web.Application([
        (r"/", MainHandler),
        (r"/celery-block", CeleryHandler),
    ], autoreload=True)

if __name__ == "__main__":
    app = make_app()
    app.listen(8000)
    tornado.ioloop.IOLoop.current().start()

celery调度端：

import os
import time
from celery import Celery
from tornado import gen

celery = Celery("tasks", broker="amqp://")
celery.conf.CELERY_RESULT_BACKEND = os.environ.get('CELERY_RESULT_BACKEND', 'amqp')

@celery.task
def sleep(seconds):
    time.sleep(float(seconds))
    return seconds

if __name__ == "__main__":
    celery.start()

Celery 的 Worker 运行在另一个进程中，独立于 Tornado 进程，不会影响 Tornado 运行效率，在处理复杂任务时候比进程模式更有效率。

 

总结

推荐使用线程和 Celery 的模式进行异步编程，轻量级的放在线程中执行，复杂的放在 Celery 中执行。当然如果有异步库使用那最好不过了。目前没有找到最佳的异步非阻塞的编程模式，可用的异步库比较局限，只有经常用的，个人编写异步库比较困难。

Python 3 中可以把 Tornado 设置为 asyncio 的模式，这样就使用 兼容 asyncio 模式的库，这应该是日后的方向。

Reference

 

《Tornado异步一》------跳转

转载于:https://www.cnblogs.com/lianzhilei/p/7821889.html

abchhcba2014
关注
AttributeError: module ‘tornado.web‘ has no attribute ‘asynchronous‘解决方案
weixin_43178406的博客
02-08 2万+
本文主要介绍了AttributeError: module ‘tornado.web’ has no attribute 'asynchronous’解决方案，希望能对新手有所帮助。 文章目录 1. 问题描述 2. 解决方案    今天在启动jupyter notebook时，出现了AttributeError: module ‘tornado.web’ has no attribute 'asynchronous’的错误提示
python学习之——tornado 实现异步调用接口 @web.asynchronous
小米mm修仙路
09-25 1656
终于找到一种 比较方便的实现 tornado 异步操作且支持多线程并发的方法了 server.py from tornado import gen, web, ioloop, template from tornado.gen import coroutine, Future from tornado.ioloop import IOLoop from tornado.concurrent import run_on_executor from concurrent.futures import Th
tornado非阻塞式Http请求AsyncHTTPClient
sunghosts的专栏
08-18 537
tornado
Tornado框架async异步调用数据库
weixin_52185313的博客
02-16 1268
Tornado框架async异步调用数据库 – 潘登同学的Tornado学习笔记 文章目录Tornado框架async异步调用数据库 -- 潘登同学的Tornado学习笔记环境搭建异步操作数据库使用aiomysql进行增查改操作ORM--peewee的使用使用方法字段类型增加数据数据查询普通查询数据筛选过滤条件模糊查询数据排序数据分页数据修改数据删除警告解决异步peewee使用方法 环境搭建 把前端代码拿过来 数据库连接上 from tornado import web from tornado imp
tornado异步协程
kuokay的博客
01-20 518
Tornado的协程 Tornado的异步编程也主要体现在网络IO的异步上，即异步Web请求。 异步Web请求客户端 Tornado提供了一个异步Web请求客户端tornado.httpclient.AsyncHTTPClient用来进行异步Web请求。 fetch(request) 用于执行一个web请求request，并异步返回一个tornado.httpclient.HTTPResponse响应。 request可以是一个url，也可以是一个tornado.httpclient.HTTPRequest
Python Tornado：构建WEB和Socket服务器并实现交互
在Python开发中，Tornado是一个强大的异步网络库，能够很好地处理Web服务器和Socket服务器的需求。在某些场景下，例如实时通信或需要服务器之间通信的应用，可能需要同时运行WEB服务器和Socket服务器。本文将介绍...
龙卷风：Tornado是Python网络框架和异步网络库，最初由FriendFeed开发
02-05
综上所述，Tornado作为一个高性能、轻量级且具备强大异步能力的Python网络框架，为开发者提供了构建高性能Web服务的工具，是Python生态系统中不可或缺的一部分。通过深入理解和熟练掌握Tornado，开发者可以构建出...
Python web框架: Tornado
justzcy的博客
06-25 1721
中文文档https://tornado-zh.readthedocs.io/zh/latest/输入命令： 备注： Tornado应该运行在类Unix平台，为了达到最佳的性能和扩展性，仅推荐Linux和BSD(充分利用Linux的epoll工具和BSD的kqueue达到高性能处理的目的)db.py main.py......
tornado-async-tcp:异步 tcp 服务器和客户端
06-21
在Python编程领域，Tornado是一...总之，Tornado提供的异步TCP功能使得在Python中开发高性能的网络应用变得简单。通过学习和实践，你可以掌握异步编程的核心概念，并将其应用到自己的项目中，提升服务的稳定性和效率。
Python的Tornado框架异步编程入门实例
09-22
Python的Tornado框架是一款轻量级、高效、非阻塞的web服务器和Python Web框架。它的异步编程模型区别于传统的...对于希望深入学习Python异步编程或者希望提升Web服务性能的开发者而言，掌握Tornado框架是很有价值的。
tornado 异步web
weixin_42528089的博客
10-20 669
若想学会异步，必先学同步。 1.同步web开发代码，就是单向进行的，服务器必须先响应前一个请求，才能处理后一个请求。 import tornado.httpserver import tornado.ioloop import tornado.options import tornado.web import tornado.httpclient import datetime from ...
Python：解决 Tornado.web 模块缺少 asynchronous 属性的问题
最新发布
BUG？不存在的！
05-24 957
在开发过程中，我们可能会遇到错误提示：AttributeError: module ‘tornado.web‘ has no attribute ‘asynchronous‘。这个错误通常是由于导入 Tornado.web 模块时缺少 asynchronous 属性引起的。以上就是解决 Tornado.web 模块缺少 asynchronous 属性的问题的两种方法，希望可以对大家有所帮助。Python：解决 Tornado.web 模块缺少 asynchronous 属性的问题。
Tornado 异步
JunChow
07-19 472
参考资料 https://www.cnblogs.com/becker/p/9335136.html Tornado依赖ioloop实现非阻塞式（同步多路IO复用）的请求响应，而ioloop采用Linux的Epoll异步网络IO模型实现。 Tornado的异步包括两个方面：异步服务器和异步客户端，无论服务器和客户端，具体的异步模式又分为回调callback和协程coroutin...
tornado之异步web服务二
dianjie9145的博客
01-02 109
前面介绍了异步方法带来的显著性能提升。我们来看下异步使用的方法。 1 首先对于get方法使用tornado.web.asynchronous进行装饰。当使用tornado.web.asynchronous进行装饰的时候，Tornado不会自己关闭连接。 你必须在你的RequestHandler对象中调用finish方法来显式地告诉Tornado关闭连接否则，请求将可能挂起，浏览器可能不会...
学习tornado -- 异步
02-28 1934
why asynchronous tornado是一个异步web framework，说是异步，是因为tornado server与client的网络交互是异步的，底层基于io event loop。但是如果client请求server处理的handler里面有一个阻塞的耗时操作，那么整体的server性能就会下降。 def MainHandler(tornado.web.Re
python学习笔记 - asynchronous@tornado 学习记录二
joy2everyone
03-15 289
继续笔记下关于tornado async一些学习知识： Tornado支持long-polling，在其提供的demo中推荐参考的例子就是那个chat demo，我这里参考chat demo，提供一个简单的long-polling测试应用“用于统计当前在线人数，并保持最新数据至各个客户端的同步更新”。 1. 通过URL加一个参数name来模拟在线用户。 2. Ajax long po...
Python开发【Tornado】：异步Web服务（一）
abchhcba2014的博客
06-21 295
异步Web服务 前言： 　　到目前为止，我们已经看到了许多使Tornado成为一个Web应用强有力框架的功能。它的简单性、易用性和便捷性使其有足够的理由成为许多Web项目的不错的选择。然而，Tornado受到最多关注的功能是其异步取得和提供内容的能力，它有着很好的理由：它使得处理非阻塞请求更容易，最终导致更高效的处理以及更好的可扩展性。在本章中，我们将看到Tornado异步请求的基础...
module ‘tornado.web‘ has no attribute ‘asynchronous‘
wendychow001的博客
02-16 413
由于学习过程中遇到tornado升级，目前的教材中的样例教旧，需要进行代码的更新
python web(异步非阻塞)框架之tornado
学习、记录、分享
11-25 173
一.相关链接 1.GitHub:https://github.com/tornadoweb/tornado 2.PyPi:https://pypi.org/project/tornado/ 3.Tornado官方文档：https://www.tornadoweb.org 4.相关教程：http://shouce.jb51.net/tornado 二.pip安装 pip install tornado 三.测试demo # -*- coding:utf-8 -*- import torna
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值