Tornado是一个异步非堵塞的web框架,但是在遇到数据库的io操作就会发现数据库的io操作是同步执行的,印象性能。
如不太了解tornado,可阅读下这篇文章 Tornado入门这一篇足以
可以三种方法可以使这个同步执行变成异步:
- 线程
- 进程
- 中间件(MQ、redis等)
编程时遇到的阻塞任务一般有两类:
- 等待 I/O 就绪(I/O 密集型),这种场景使用
ThreadPoolExecutor
- 耗时的计算工作(CPU 密集型),这种场景使用
ProcessPoolExecutor
1.线程
1-1.装饰器方式一
@run_on_executor
和ThreadPoolExecutor
配合使用定义同步代码async
和await
配合调用装饰后的方法
import time
from concurrent.futures.thread import ThreadPoolExecutor
from tornado import web, ioloop
from tornado.concurrent import run_on_executor
class SyncToAsyncThreadHandler(web.RequestHandler):
executor = ThreadPoolExecutor(max_workers=2)
@run_on_executor
def sleep(self):
print("休息1...start")
time.sleep(5)
print("休息1...end")
return 'ok'
async def get(self):
res = await self.sleep()
self.write(res)
url_map = [
("/?", SyncToAsyncThreadHandler)
]
if __name__ == '__main__':
app = web.Application(url_map, debug=True)
app.listen(8888)
print('started...')
ioloop.IOLoop.current().start()
1-2.装饰器方式二
@run_on_executor
和ThreadPoolExecutor
配合使用定义同步代码@gen.coroutine
和yield
配合调用装饰后的方法
import time
from concurrent.futures.thread import ThreadPoolExecutor
from tornado import gen, web, ioloop
from tornado.concurrent import run_on_executor
class SyncToAsyncThreadHandler(web.RequestHandler):
executor = ThreadPoolExecutor(max_workers=2)
@run_on_executor
def sleep(self):
print("休息1...start")
time.sleep(5)
print("休息1...end")
return 'ok'
@gen.coroutine
def get(self):
res = yield self.sleep()
self.write(res)
url_map = [
("/?", SyncToAsyncThreadHandler)
]
if __name__ == '__main__':
app = web.Application(url_map, debug=True)
app.listen(8888)
print('started...')
ioloop.IOLoop.current().start()
1-3.submit方式
可以使用executor.submit
来提交函数
executor除了
submit
外,还有map
方法,它们的返回值是Future,或者对Futureadd_done_callback
的方法回调方式处理
import time
from concurrent.futures.thread import ThreadPoolExecutor
from tornado import gen, web, ioloop
executor = ThreadPoolExecutor(max_workers=2)
class SyncToAsyncThreadHandler(web.RequestHandler):
def sleep(self):
print("休息1...start")
time.sleep(5)
print("休息1...end")
return 'ok'
@gen.coroutine
def get(self):
res = yield executor.submit(self.sleep)
self.write(res)
url_map = [
("/?", SyncToAsyncThreadHandler)
]
if __name__ == '__main__':
app = web.Application(url_map, debug=True)
app.listen(8888)
print('started...')
ioloop.IOLoop.current().start()
这
executor.submit
使用async和await方式无效
1-4.ioloop方式一
- 使用ioloop的IOLoop的run_in_executor函数,指定传入executor和要调用的同步函数
- 用
@gen.coroutine
和yield
装饰成异步函数
import time
from concurrent.futures.thread import ThreadPoolExecutor
from tornado import gen, web, ioloop
executor = ThreadPoolExecutor(max_workers=2)
class SyncToAsyncThreadHandler(web.RequestHandler):
def sleep(self):
print("休息1...start")
time.sleep(5)
print("休息1...end")
return 'ok'
@gen.coroutine
def get(self):
rest = yield ioloop.IOLoop.current().run_in_executor(executor, self.sleep)
self.write(rest)
url_map = [
("/?", SyncToAsyncThreadHandler)
]
if __name__ == '__main__':
app = web.Application(url_map, debug=True)
app.listen(8888)
print('started...')
ioloop.IOLoop.current().start()
1-5.ioloop方式二
- 跟ioloop方式一类似,只不过用
async
和await
方式
import time
from concurrent.futures.thread import ThreadPoolExecutor
from tornado import gen, web, ioloop
executor = ThreadPoolExecutor(max_workers=2)
class SyncToAsyncThreadHandler(web.RequestHandler):
def sleep(self):
print("休息1...start")
time.sleep(5)
print("休息1...end")
return 'ok'
async def get(self):
rest = await ioloop.IOLoop.current().run_in_executor(executor, self.sleep)
self.write(rest)
url_map = [
("/?", SyncToAsyncThreadHandler)
]
if __name__ == '__main__':
app = web.Application(url_map, debug=True)
app.listen(8888)
print('started...')
ioloop.IOLoop.current().start()
2.进程
2-1.submit方式
import time
from concurrent.futures.process import ProcessPoolExecutor
from tornado import gen, web, ioloop
executor = ProcessPoolExecutor(max_workers=2)
def sleep():
print("休息...start")
time.sleep(5)
print("休息...end")
return 'ok'
class SyncToAsyncProcessHandler(web.RequestHandler):
@gen.coroutine
def get(self):
rest = yield executor.submit(sleep)
self.write(rest)
url_map = [
("/?", SyncToAsyncProcessHandler)
]
if __name__ == '__main__':
app = web.Application(url_map, debug=True)
app.listen(8888)
print('started...')
ioloop.IOLoop.current().start()
2-2.ioloop方式
import time
from concurrent.futures.process import ProcessPoolExecutor
from tornado import gen, web, ioloop
executor = ProcessPoolExecutor(max_workers=2)
def sleep():
print("休息...start")
time.sleep(5)
print("休息...end")
return 'ok'
class SyncToAsyncProcessHandler(web.RequestHandler):
async def get(self):
rest = await ioloop.IOLoop.current().run_in_executor(executor, sleep)
self.write(rest)
url_map = [
("/?", SyncToAsyncProcessHandler)
]
if __name__ == '__main__':
app = web.Application(url_map, debug=True)
app.listen(8888)
print('started...')
ioloop.IOLoop.current().start()
3.中间件
基于AMQP and Redis实现的,所以要安装相关软件,有兴趣可以去阅读下说明,很简单
4.网站
个人博客刚开不久,主要用来辅助手册,写些零碎的知识点
注册下个人用户,就可以管理自己的书签/链接、享用各类学习手册,主要用来写手册,分享学习。