pyhton greent hread

通过学习openstack源码,发现os代码里服务线程大都是green thread,这是个什么东西?我们先看看os里怎么用的:

以nova-api服务为例,这是一个提供rest api接口的服务,也就是为用户系统所使用,这个模块在os代码里由wsgi.py提供,下边是wsgi模块的Server定义:

class Server(object):
    """Server class to manage multiple WSGI sockets and applications."""

    def __init__(self, threads=1000):
        logging.basicConfig()
        self.pool = eventlet.GreenPool(threads)

    def start(self, application, port, host='0.0.0.0', backlog=128):
        """Run a WSGI server with the given application."""
        arg0 = sys.argv[0]
        logging.audit(_("Starting %(arg0)s on %(host)s:%(port)s") % locals())
        socket = eventlet.listen((host, port), backlog=backlog)
        self.pool.spawn_n(self._run, application, socket)

    def wait(self):
        """Wait until all servers have completed running."""
        try:
            self.pool.waitall()
        except KeyboardInterrupt:
            pass

    def _run(self, application, socket):
        """Start a WSGI server in a new green thread."""
        logger = logging.getLogger('eventlet.wsgi.server')
        eventlet.wsgi.server(socket, application, custom_pool=self.pool,
                             log=WritableLogger(logger))

如类的介绍,这是一个管理sockets和应用程序的类,那这个类是如何实现的呢?我们先看看该类的构造函数,类实例化时有一个threads参数,传给eventlet.GreenPool,这是干什么呢?eventlet又是什么?

eventlet

官方的解释:

Eventlet is a concurrent networking library for Python that allows you to change how you run your code, not how you write it.

It uses epoll or libevent for highly scalable non-blocking I/O. Coroutines ensure that the developer uses a blocking style of programming that is similar to threading, but provide the benefits of non-blocking I/O. The event dispatch is implicit, which means you can easily use Eventlet from the Python interpreter, or as a small part of a larger application.

It’s easy to get started using Eventlet, and easy to convert existing applications to use it.

关键几点:一个协程库,依赖epoll或者libevent,实现高可扩展的非阻塞网络io程序。

安装方式:pip install eventlet

我们来看几个例子:

>>> import eventlet
>>> from eventlet.green import urllib2
>>> gt = eventlet.spawn(urllib2.urlopen, 'https://www.baidu.com')
>>> gt.wait()
<addinfourl at 27997320 whose fp = <socket._fileobject object at 0x1a597d0>>
>>> a=gt.wait()
>>> dir(a)
['__doc__', '__init__', '__iter__', '__module__', '__repr__', 'close', 'code', 'fileno', 'fp', 'getcode', 'geturl', 'headers', 'info', 'msg', 'next', 'read', 'readline', 'readlines', 'url']
>>> a.rul
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: addinfourl instance has no attribute 'rul'
>>> a.url
'https://www.baidu.com'
>>> a.headers
<httplib.HTTPMessage instance at 0x1aadb00>
>>> a.msg
'OK'
>>> a.read()
'<html>\r\n<head>\r\n\t<script>\r\n\t\tlocation.replace(location.href.replace("https://","http://"));\r\n\t</script>\r\n</head>\r\n<body>\r\n\t<noscript><meta http-equiv="refresh" content="0;url=http://www.baidu.com/"></noscript>\r\n</body>\r\n</html>'
>>>
>>>

再看:

#!/usr/bin/env python
"""
This is a simple web "crawler" that fetches a bunch of urls using a pool to
control the number of outbound connections.
"""
import eventlet
from eventlet.green import urllib2


urls = [
    "https://timgsa.baidu.com/timg?image&quality=80&size=b9999_10000&sec=1587273826274&di=37d7fa5daa3f9c8ff99aed93329c07ff&imgtype=0&src=http%3A%2F%2Fimg.aso.aizhan.com%2Ficon%2F42%2F53%2Ff7%2F4253f7e9545c2293b6914705a41bfe51.jpg",
    "http://python.org/images/python-logo.gif",
]


def fetch(url):
    print("opening", url)
    body = urllib2.urlopen(url).read()
    print("done with", url)
    return url, body


pool = eventlet.GreenPool(200)
for url, body in pool.imap(fetch, urls):
    print("got body from", url, "of length", len(body))
运行输出:

('opening', 'https://timgsa.baidu.com/timg?image&quality=80&size=b9999_10000&sec=1587273826274&di=37d7fa5daa3f9c8ff99aed93329c07ff&imgtype=0&src=http%3A%2F%2Fimg.aso.aizhan.com%2Ficon%2F42%2F53%2Ff7%2F4253f7e9545c2293b6914705a41bfe51.jpg')
('opening', 'http://python.org/images/python-logo.gif')
('done with', 'https://timgsa.baidu.com/timg?image&quality=80&size=b9999_10000&sec=1587273826274&di=37d7fa5daa3f9c8ff99aed93329c07ff&imgtype=0&src=http%3A%2F%2Fimg.aso.aizhan.com%2Ficon%2F42%2F53%2Ff7%2F4253f7e9545c2293b6914705a41bfe51.jpg')
('got body from', 'https://timgsa.baidu.com/timg?image&quality=80&size=b9999_10000&sec=1587273826274&di=37d7fa5daa3f9c8ff99aed93329c07ff&imgtype=0&src=http%3A%2F%2Fimg.aso.aizhan.com%2Ficon%2F42%2F53%2Ff7%2F4253f7e9545c2293b6914705a41bfe51.jpg', 'of length', 10452)
('done with', 'http://python.org/images/python-logo.gif')
('got body from', 'http://python.org/images/python-logo.gif', 'of length', 2549)
[root@localhost python]#

发现是顺序执行的,那如何启动一个wsgi server呢?

官方给的是这样的:

"""This is a simple example of running a wsgi application with eventlet.
For a more fully-featured server which supports multiple processes,
multiple threads, and graceful code reloading, see:

http://pypi.python.org/pypi/Spawning/
"""

import eventlet
from eventlet import wsgi


def hello_world(env, start_response):
    if env['PATH_INFO'] != '/':
        start_response('404 Not Found', [('Content-Type', 'text/plain')])
        return ['Not Found\r\n']
    start_response('200 OK', [('Content-Type', 'text/plain')])
    return ['Hello, World!\r\n']

wsgi.server(eventlet.listen(('', 8090)), hello_world)

运行:
[root@localhost python]# python eventlet-1.py
(17415) wsgi starting up on http://0.0.0.0:8090
(17415) accepted ('172.30.59.209', 57684)
172.30.59.209 - - [19/Apr/2020 10:59:10] "GET / HTTP/1.1" 200 140 0.000132
172.30.59.209 - - [19/Apr/2020 10:59:11] "GET / HTTP/1.1" 200 140 0.000099
172.30.59.209 - - [19/Apr/2020 10:59:12] "GET / HTTP/1.1" 200 140 0.000105
^Cwsgi exiting
(17415) wsgi exited, is_accepting=True

这是一个简单的例子,我们看到基本的用法就是这样,server函数需要一个socket参数,一个app参数,这个app需要有两个参数,一个是env环境参数,一个是用来处理http响应的函数。,默认监听所有地址。

 

  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值