Keystone在创建Web服务器对外提供服务时,使用了第三方Eventlet库。
一.基本概念
官方网站对eventlet的描述是:
Eventlet is built around the concept of green threads (i.e. coroutines, we use the terms interchangeably) that are launched to do network-related work. Green threads differ from normal threads in two main ways:
- Green threads are so cheap they are nearly free. You do not have to conserve green threads like you would normal threads. In general, there will be at least one green thread per network connection.
- Green threads cooperatively yield to each other instead of preemptively being scheduled. The major advantage from this behavior is that shared data structures don’t need locks, because only if a yield is explicitly called can another green thread have access to the data structure. It is also possible to inspect primitives such as queues to see if they have any pending data.
基础API说明:
1.greenthread spawn
eventlet.spawn(func,*args,**kw) 开一个greenthread跑func
eventlet.spawn_n(func,*args,**kw)开一个greenthread跑func,但不知道greenthread何时终止,运行速度快
eventlet.spawn_after(seconds,func,*args,**kw)开一个greenthread跑func
2.greenthread control
eventlet.sleep(seconds=0) 睡眠当前greenthread
classeventlet.GreenPool 线程池管理并发线程
classeventlet.GreenPile 能批量处理大量工作
classeventlet.Queue 线程间通信
classeventlet.Timeout 线程超时处理,raise exceptions after timeout seconds.
3.patching functions
eventlet.import_patched(modulename,*additional_modules,**kw_additional_modules) 向线程导入标准模块的绿色版本
eventlet.monkey_patch(all=True,os=False,select=False,socket=False,thread=False,time=False) 全局向线程导入系统模块
4.network convenience functions
eventlet.connect(addr, family=2, bind=None) 客户端sockets,连接对象
eventlet.listen(addr,family=2, backlog=50) 服务端sockes,监听对象
eventlet.wrap_ssl(sock,*a, **kw) 配置ssl,通过PyOpenssl
eventlet.serve(sock,handle, concurrency=1000) 建立服务器,scck为listen对象,handle为处理函数,最后一个为最大并发支持数
class eventlet.StopServe 停止服务器
二.Eventlet设计模式
有三种模式client,server和dispatch模式
client模式(Wrab crawler网页爬虫),测试urls有错误
urls = ["http://www.google.com/intl/en_ALL/images/logo.gif",
"https://wiki.secondlife.com/w/images/secondlife.jpg",
"http://us.i1.yimg.com/us.yimg.com/i/ww/beta/y3.gif"]
import eventlet
from eventlet.green import urllib2
def fetch(url):
return urllib2.urlopen(url).read()
pool = eventlet.GreenPool()
for body in pool.imap(fetch, urls):
print "got body", len(body)
server模式,建立一个简单的server,接受一个字符,返回一个字符
import eventlet
def handle(client):
while True:
c = client.recv(1)
if not c: break
client.sendall(c)
server = eventlet.listen(('0.0.0.0', 6000))
pool = eventlet.GreenPool(10000)
while True:
new_sock, address = server.accept()
pool.spawn_n(handle, new_sock)
dispatch模式,处理请求多个RSS feeds,处理完,返回多个RSS feeds的数据titile
import eventlet
feedparser = eventlet.import_patched('feedparser')
pool = eventlet.GreenPool()
def fetch_title(url):
d = feedparser.parse(url)
return d.feed.get('title', '')
def app(environ, start_response):
pile = eventlet.GreenPile(pool)
for url in environ['wsgi.input'].readlines():
pile.spawn(fetch_title, url)
titles = '\n'.join(pile)
start_response('200 OK', [('Content-type', 'text/plain')])
return [titles]
三.Eventlet测试
客户端-服务器程序,发送数据,并返回发送的数据
客户端代码:
import eventlet
c=eventlet.connect(('127.0.0.1', 6000))
while True:
data=raw_input('Enter data:')
c.sendall(data)
rc=c.recv(1024)
print rc
服务端代码:
import eventlet
c=eventlet.connect(('127.0.0.1', 6000))
while True:
data=raw_input('Enter data:')
c.sendall(data)
rc=c.recv(1024)
print rc
import eventlet
def handle(client):
while True:
c = client.recv(1024)
print c
client.sendall(c)
server = eventlet.listen(('0.0.0.0', 6000))
pool = eventlet.GreenPool(10000)
while True:
new_sock, address = server.accept()
pool.spawn_n(handle, new_sock)
WSGI服务器实现
import eventlet from eventlet import wsgi def hello_world(env, start_response): if env['PATH_INFO'] != '/': start_response('404 Not Found', [('Content-Type', 'text/plain')]) return ['Not Found\r\n' eventlet.import_patched('httplib2') eventlet.monkey_patch(socket=True, select=True) start_response('200 OK', [('Content-Type', 'text/plain')]) return ['Hello, World!\r\n'] wsgi.server(eventlet.listen(('', 8090)), hello_world)
四.标准库在eventlet下使用
通过Import Green导入单个,或通过Monkeypatching the Standard Library导入所有的库
eventlet.import_patched('httplib2')
eventlet.monkey_patch(socket=True, select=True)
五.eventet相关模块
- backdoor – Python interactive interpreter within a running process
- corolocal – Coroutine local storage
- debug – Debugging tools for Eventlet
- db_pool – DBAPI 2 database connection pooling
- event – Cross-greenthread primitive
- greenpool – Green Thread Pools
- greenthread – Green Thread Implementation
- pools - Generic pools of resources
- queue – Queue class
- semaphore – Semaphore classes
- timeout – Universal Timeouts
- websocket – Websocket Server
- wsgi – WSGI server
- eventlet.green.zmq – ØMQ support
- zmq – The pyzmq ØMQ python bindings
2013/10/27
21:31