python http服务器性能,Python BaseHTTPServer性能测试

最近我要在短时间内要做一个内部使用的诊断服务，于是想到了使用Python的BaseHTTPServer。Python开发速度是飞快的。等开发完了，回过头想了想，发现这个以后性能会是一个瓶颈。诊断服务要调用很多别的服务，单线程的话QPS可能会只有一个，必须得用多线程。于是使用mixin将服务改成多线程版。

事后还是不太放心，于是使用abench分别压了两个版本的服务。服务代码如下，非常简单，看看空转速度怎么样。下面测试使用的是Python 2.6.5版本。如果用2.4.3，性能比2.6.5要差。#!/usr/bin/env python

from BaseHTTPServer import HTTPServer, BaseHTTPRequestHandler

from SocketServer import ThreadingMixIn

import threading

class Handler(BaseHTTPRequestHandler):

def do_GET(self):

self.send_response(200)

self.end_headers()

message = threading.currentThread().getName()

self.wfile.write(message)

self.wfile.write('\n')

return

#class ThreadedHTTPServer(ThreadingMixIn, HTTPServer):

# """Handle requests in a separate thread."""

class ThreadedHTTPServer(HTTPServer):

"""Handle requests in a separate thread."""

if __name__ == '__main__':

server = ThreadedHTTPServer(('localhost', 8181), Handler)

print 'Starting server, use to stop'

server.serve_forever()

一、单线程版本。单线程的情况下，能将一个CPU压满。在本机上压，latency居然到2.48ms，还是有点高。CMD: abench_2.0.7_release -p 40 -s 60 localhost 8181 /home/henshao/1.log

QPS: 2905.02

Latency: 2.48 ms

Query Success Number: 174301

Connect Failed Number: 0

Query Failed Number: 917

Query Timeout Number: 13

Match Failed Number: 0

No DocsReturn Number: 174301 (100.00%)

Average DocsReturn Number: 0.00

No DocsFound Number: 174301 (100.00%)

Average DocsFound Number: 0.00

Average Return Length(Bytes): 11

Min Response Time: 0.5 ms

Max Response Time: 3017.9 ms

25 Percentile: 2.1 ms

50 Percentile: 2.2 ms

75 Percentile: 2.3 ms

90 Percentile: 2.4 ms

95 Percentile: 2.5 ms

99 Percentile: 2.6 ms

二、多线程版本。悲剧的多线程，性能不见提升，反而下降了。Python的伪多线程，导致服务也只能把一个CPU压满。由于这个服务完全是空转的，没有IO，所以和真实情况不同。只能寄希望于在有较多IO的情况下，性能能有所提升。CMD: abench_2.0.7_release -p 40 -s 60 localhost 8181 /home/henshao/1.log

QPS: 1449.63

Latency: 4.42 ms

Query Success Number: 86978

Connect Failed Number: 0

Query Failed Number: 168653

Query Timeout Number: 0

Match Failed Number: 0

No DocsReturn Number: 86978 (100.00%)

Average DocsReturn Number: 0.00

No DocsFound Number: 86978 (100.00%)

Average DocsFound Number: 0.00

Average Return Length(Bytes): 12

Min Response Time: 0.6 ms

Max Response Time: 1410.9 ms

25 Percentile: 3.0 ms

50 Percentile: 3.4 ms

75 Percentile: 3.8 ms

90 Percentile: 4.3 ms

95 Percentile: 4.6 ms

99 Percentile: 5.5 ms

多线程情况下CPU使用率如下图所示。

Python多线程情况下，要使用TLS。在C/C++中可以使用__thread直接声明一个变量为私有变量(这个特性解决了我使用ctypes面临的多线程问题)，Python则稍微麻烦点。下面的代码是一个测试的例子。有一个诡异之处是TestThread::__init__居然是主线程调用的，因此这里设置的私有变量在run里面是不能用的。(PS: 类成员变量使用TLS真是愚蠢之至)from threading import local, enumerate, Thread, currentThread

class TestThread(Thread):

def __init__ (self):

Thread.__init__(self)

self.local_data = local()

self.local_data.name = 'abc'

self.local_data.dict = {}

self.local_data.dict['a'] = 'b'

print '++++', currentThread(), id(self.local_data.dict), self.local_data.__dict__

def run(self):

self.local_data.name = 'abc'

self.local_data.dict = {}

self.local_data.dict['a'] = 'b'

print self.local_data.__dict__

print '----', currentThread(), id(self.local_data.dict), self.local_data.__dict__

if __name__ == '__main__':

t1 = TestThread()

t1.start()

t1.join()

t2 = TestThread()

t2.start()

t2.join()