Python的tornado框架性能研究
1. Summary
Tornado框架算是Python Web框架中比较有名的一个。以前一直没空研究，最近因为要做个项目，干脆深入考察一下tornado的性能。
2. 测试环境
2.1 软硬件环境
| key | value |
|---|---|
| CPU【腾讯云VPS虚拟机】 | 8 核心 Intel(R) Xeon(R) CPU E5-26xx v3 |
| 内存 | 8GB |
| 硬盘 | 100GB |
| 操作系统 | Ubuntu Server 16.04LTS 64bit |
| python | 2.7.12 64bit |
| tornado | 4.4.2 |
| pycurl | 7.43.0 |
| nginx | 1.10.0 |
| ab【包含在apache2-utils里的一个压测命令】 | 2.3 |
2.2 关于ab
考虑到我们即将要压测，而现在Linux系统大多都有并发连接数限制，所以我们必须要做下面几件事
- 执行以下命令，使得Linux下单进程可以打开的文件句柄数大于1024
ulimit -u 40960 - 编辑/etc/sysctl.conf文件
设置net.ipv4.tcp_syncookies = 0
2.3 测试代码
# -.- coding:utf-8 -.-
import platform
import tornado.ioloop
import tornado.web
import tornado.gen
import tornado.options
import select
import tornado.httpserver
import tornado.httpclient
import tornado.concurrent
__author__ = 'chenjm'
class BlockHandler(tornado.web.RequestHandler):
@tornado.web.asynchronous # 重要！这个装饰器使得当前请求变成长连接，直到调用self.finish()为止
@tornado.gen.coroutine # 重要！ tornado的重要装饰器，使得当前请求可以用yield异步执行IO操作
def get(self):
yield tornado.gen.sleep(3) # 模拟长达3秒的IO操作，但是不阻塞主线程
self.write("i sleep 3s\n")
self.finish() # 结束本次连接
class AsyncBlockHandler(tornado.web.RequestHandler):
@tornado.web.asynchronous
@tornado.gen.coroutine
def get(self, *args, **kwargs):
http_client = tornado.httpclient.AsyncHTTPClient() # tornado异步客户端
http_req = tornado.httpclient.HTTPRequest(
url='http://127.0.0.1:8000/block',
method='GET',
validate_cert=False,
request_timeout=30)
response = yield tornado.gen.Task(http_client.fetch, http_req) # 执行异步HTTP请求
if response.code == 200:
self.write(response.body)
else:
print "error code = ", response.code
self.finish()
class TestHandler(tornado.web.RequestHandler):
@tornado.web.asynchronous
@tornado.gen.coroutine
def get(self, *args, **kwargs):
self.write("Hello Word")
self.finish()
class Application(tornado.web.Application):
def __init__(self):
handlers = [
('/block', BlockHandler),
('/async_block', AsyncBlockHandler),
('/test', TestHandler),
]
super(Application, self).__init__(handlers)
if __name__ == "__main__":
# tornado框架可以解析命令行参数，如果有这一句话，那么就可以传入--port=XX参数来指定socket绑定端口
tornado.options.define("port", default=8000, help="run on the given port", type=int)
# 分析命令行
tornado.options.parse_command_line()
# 生成http服务器对象
http_server = tornado.httpserver.HTTPServer(Application())
print u"Current System:", platform.system()
# 可以看看是否启用了epoll（仅对Linux系统有效）
print "epoll enabled : ", hasattr(select, "epoll")
# 如果不写这句话，那么tornado会使用默认的异步HTTP客户端，作者测试过，性能比较低。
# 所以最好按照官方文档给出的做法，使用基于libcurl的客户端框架，但是前提是安装pycurl
tornado.httpclient.AsyncHTTPClient.configure("tornado.curl_httpclient.CurlAsyncHTTPClient")
# tornado默认异步客户端最大并发是10，所以要把这个开大点，否则影响并发
# 注意！如果你启用了上述libcurl客户端，那么这句话一定要写在后面
tornado.httpclient.AsyncHTTPClient.configure(None, max_clients=20000)
# 侦听socket端口
http_server.listen(tornado.options.options.port)
# 启动事件循环
tornado.ioloop.IOLoop.current().start()
3. Tornado性能考察
3.1 Hello World！
我们先来测试一下test接口
3.1.1 1000个客户端，总共请求1000次
执行命令：【注意：不带-k参数】
ab -c 1000 -n 1000 "http://127.0.0.1:8000/test"结果：
Document Path: /test
Document Length: 10 bytes
Concurrency Level: 1000
Time taken for tests: 0.954 seconds
Complete requests: 1000
Failed requests: 0
Total transferred: 205000 bytes
HTML transferred: 10000 bytes
Requests per second: 1048.56 [#/sec] (mean)
Time per request: 953.685 [ms] (mean)
Time per request: 0.954 [ms] (mean, across all concurrent requests)
Transfer rate: 209.92 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 2 4.9 0 15
Processing: 1 110 39.7 105 799
Waiting: 1 110 39.7 105 799
Total: 14 113 39.1 105 799
Percentage of the requests served within a certain time (ms)
50% 105
66% 114
75% 118
80% 143
90% 146
95% 167
98% 182
99% 187
100% 799 (longest request)我们发现，大部分请求在200ms之内都被处理完毕，tornado应付得轻轻松松
3.1.2 2000个客户端，总共请求2000次
执行命令：
ab -c 2000 -n 2000 "http://127.0.0.1:8000/test"结果：
Document Path: /test
Document Length: 10 bytes
Concurrency Level: 2000
Time taken for tests: 1.895 seconds
Complete requests: 2000
Failed requests: 0
Total transferred: 410000 bytes
HTML transferred: 20000 bytes
Requests per second: 1055.60 [#/sec] (mean)
Time per request: 1894.649 [ms] (mean)
Time per request: 0.947 [ms] (mean, across all concurrent requests)
Transfer rate: 211.33 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 76 260.1 0 1001
Processing: 1 164 180.8 105 888
Waiting: 1 164 180.8 105 888
Total: 21 240 429.3 105 1887
Percentage of the requests served within a certain time (ms)
50% 105
66% 108
75% 125
80% 186
90% 260
95% 1814
98% 1861
99% 1875
100% 1887 (longest request)我们发现90%的请求在260ms之内得到相应，但是“Requests per second”仍然维持在1000次左右，也许这是个瓶颈。
3.1.3 2000个客户端，总共请求20000次（模拟用于通过浏览器访问网站）
执行命令：【注意，这里带-k开关， 因为现在大多数浏览器都是keep_alive访问网站】
ab -c 2000 -n 20000 -k "http://127.0.0.1:8000/test"结果：
Document Path: /test
Document Length: 10 bytes
Concurrency Level: 2000
Time taken for tests: 13.474 seconds
Complete requests: 20000
Failed requests: 0
Keep-Alive requests: 20000
Total transferred: 4580000 bytes
HTML transferred: 200000 bytes
Requests per second: 1484.34 [#/sec] (mean)
Time per request: 1347.402 [ms] (mean)
Time per request: 0.674 [ms] (mean, across all concurrent requests)
Transfer rate: 331.95 [Kbytes/sec] received
Connection Times (ms)
min mean[+/-sd] median max
Connect: 0 110 773.9 0 7011
Processing: 1 461 145.0 452 908
Waiting: 1 461 145.0 452 908
Total: 22 572 789.7 464 7919
Percentage of the requests served within a certain time (ms)
50% 464
66% 559
75% 604
80% 610
90% 620
95% 625
98% 1547
99% 7286
100% 7919 (longest request)
我们发现95%的请求在625ms之内得到相应，“Requests per second”维持在1500次左右，此时有很多连接的“Connect”花了很长时间，这说明，Tornado针对“listen+accept”操作的速度趋向于饱和。
3.2 连接保持很长的时间
未完待续。。。
to be continued……
扫一扫
947
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
