通过学习openstack源码,发现os代码里服务线程大都是green thread,这是个什么东西?我们先看看os里怎么用的:
以nova-api服务为例,这是一个提供rest api接口的服务,也就是为用户系统所使用,这个模块在os代码里由wsgi.py提供,下边是wsgi模块的Server定义:
class Server(object):
"""Server class to manage multiple WSGI sockets and applications."""
def __init__(self, threads=1000):
logging.basicConfig()
self.pool = eventlet.GreenPool(threads)
def start(self, application, port, host='0.0.0.0', backlog=128):
"""Run a WSGI server with the given application."""
arg0 = sys.argv[0]
logging.audit(_("Starting %(arg0)s on %(host)s:%(port)s") % locals())
socket = eventlet.listen((host, port), backlog=backlog)
self.pool.spawn_n(self._run, application, socket)
def wait(self):
"""Wait until all servers have completed running."""
try:
self.pool.waitall()
except KeyboardInterrupt:
pass
def _run(self, application, socket):
"""Start a WSGI server in a new green thread."""
logger = logging.getLogger('eventlet.wsgi.server')
eventlet.wsgi.server(socket, application, custom_pool=self.pool,
log=WritableLogger(logger))
如类的介绍,这是一个管理sockets和应用程序的类,那这个类是如何实现的呢?我们先看看该类的构造函数,类实例化时有一个threads参数,传给eventlet.GreenPool,这是干什么呢?eventlet又是什么?
eventlet
官方的解释:
Eventlet is a concurrent networking library for Python that allows you to change how you run your code, not how you write it.
It uses epoll or libevent for highly scalable non-blocking I/O. Coroutines ensure that the developer uses a blocking style of programming that is similar to threading, but provide the benefits of non-blocking I/O. The event dispatch is implicit, which means you can easily use Eventlet from the Python interpreter, or as a small part of a larger application.
It’s easy to get started using Eventlet, and easy to convert existing applications to use it.
关键几点:一个协程库,依赖epoll或者libevent,实现高可扩展的非阻塞网络io程序。
安装方式:pip install eventlet
我们来看几个例子:
>>> import eventlet
>>> from eventlet.green import urllib2
>>> gt = eventlet.spawn(urllib2.urlopen, 'https://www.baidu.com')
>>> gt.wait()
<addinfourl at 27997320 whose fp = <socket._fileobject object at 0x1a597d0>>
>>> a=gt.wait()
>>> dir(a)
['__doc__', '__init__', '__iter__', '__module__', '__repr__', 'close', 'code', 'fileno', 'fp', 'getcode', 'geturl', 'headers', 'info', 'msg', 'next', 'read', 'readline', 'readlines', 'url']
>>> a.rul
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
AttributeError: addinfourl instance has no attribute 'rul'
>>> a.url
'https://www.baidu.com'
>>> a.headers
<httplib.HTTPMessage instance at 0x1aadb00>
>>> a.msg
'OK'
>>> a.read()
'<html>\r\n<head>\r\n\t<script>\r\n\t\tlocation.replace(location.href.replace("https://","http://"));\r\n\t</script>\r\n</head>\r\n<body>\r\n\t<noscript><meta http-equiv="refresh" content="0;url=http://www.baidu.com/"></noscript>\r\n</body>\r\n</html>'
>>>
>>>
再看:
#!/usr/bin/env python
"""
This is a simple web "crawler" that fetches a bunch of urls using a pool to
control the number of outbound connections.
"""
import eventlet
from eventlet.green import urllib2
urls = [
"https://timgsa.baidu.com/timg?image&quality=80&size=b9999_10000&sec=1587273826274&di=37d7fa5daa3f9c8ff99aed93329c07ff&imgtype=0&src=http%3A%2F%2Fimg.aso.aizhan.com%2Ficon%2F42%2F53%2Ff7%2F4253f7e9545c2293b6914705a41bfe51.jpg",
"http://python.org/images/python-logo.gif",
]
def fetch(url):
print("opening", url)
body = urllib2.urlopen(url).read()
print("done with", url)
return url, body
pool = eventlet.GreenPool(200)
for url, body in pool.imap(fetch, urls):
print("got body from", url, "of length", len(body))
运行输出:
('opening', 'https://timgsa.baidu.com/timg?image&quality=80&size=b9999_10000&sec=1587273826274&di=37d7fa5daa3f9c8ff99aed93329c07ff&imgtype=0&src=http%3A%2F%2Fimg.aso.aizhan.com%2Ficon%2F42%2F53%2Ff7%2F4253f7e9545c2293b6914705a41bfe51.jpg')
('opening', 'http://python.org/images/python-logo.gif')
('done with', 'https://timgsa.baidu.com/timg?image&quality=80&size=b9999_10000&sec=1587273826274&di=37d7fa5daa3f9c8ff99aed93329c07ff&imgtype=0&src=http%3A%2F%2Fimg.aso.aizhan.com%2Ficon%2F42%2F53%2Ff7%2F4253f7e9545c2293b6914705a41bfe51.jpg')
('got body from', 'https://timgsa.baidu.com/timg?image&quality=80&size=b9999_10000&sec=1587273826274&di=37d7fa5daa3f9c8ff99aed93329c07ff&imgtype=0&src=http%3A%2F%2Fimg.aso.aizhan.com%2Ficon%2F42%2F53%2Ff7%2F4253f7e9545c2293b6914705a41bfe51.jpg', 'of length', 10452)
('done with', 'http://python.org/images/python-logo.gif')
('got body from', 'http://python.org/images/python-logo.gif', 'of length', 2549)
[root@localhost python]#
发现是顺序执行的,那如何启动一个wsgi server呢?
官方给的是这样的:
"""This is a simple example of running a wsgi application with eventlet.
For a more fully-featured server which supports multiple processes,
multiple threads, and graceful code reloading, see:
http://pypi.python.org/pypi/Spawning/
"""
import eventlet
from eventlet import wsgi
def hello_world(env, start_response):
if env['PATH_INFO'] != '/':
start_response('404 Not Found', [('Content-Type', 'text/plain')])
return ['Not Found\r\n']
start_response('200 OK', [('Content-Type', 'text/plain')])
return ['Hello, World!\r\n']
wsgi.server(eventlet.listen(('', 8090)), hello_world)
运行:
[root@localhost python]# python eventlet-1.py
(17415) wsgi starting up on http://0.0.0.0:8090
(17415) accepted ('172.30.59.209', 57684)
172.30.59.209 - - [19/Apr/2020 10:59:10] "GET / HTTP/1.1" 200 140 0.000132
172.30.59.209 - - [19/Apr/2020 10:59:11] "GET / HTTP/1.1" 200 140 0.000099
172.30.59.209 - - [19/Apr/2020 10:59:12] "GET / HTTP/1.1" 200 140 0.000105
^Cwsgi exiting
(17415) wsgi exited, is_accepting=True
这是一个简单的例子,我们看到基本的用法就是这样,server函数需要一个socket参数,一个app参数,这个app需要有两个参数,一个是env环境参数,一个是用来处理http响应的函数。,默认监听所有地址。