python异步io 队列_【Python复习巩固 Day10】异步IO

异步IO：当代码需要执行一个耗时的IO操作时，它只发出IO指令，并不等待IO结果，然后就去执行其他代码了。一段时间后，当IO返回结果时，再通知CPU进行处理

异步IO模型需要一个消息循环，在消息循环中，主线程不断重复“读取消息-处理消息”这个过程

在一个消息模型中，处理一个消息必须非常迅速，否则，主线程将无法及时处理消息队列中的其他消息，导致程序看上去停止响应

在异步IO模型中，一个线程可以同时处理多个IO请求，并且没有切换线程的操作，对于大多数IO密集型的应用程序，使用异步IO将大大提升系统的多任务处理能力

协程

协程，又称微线程，协程看上去也是子程序，但执行过程中，在子程序内部可以中断，然后转而执行别的子程序，在适当的时候再返回来接着执行

基于协程的生产者-消费者模型

1、首先，调用send(None)启动生成器

2、然后，一旦生产了商品，通过send(n)切换到consumer消费

3、consumer通过yield拿到消息，处理消息，又通过yield把结果传回

4、producer拿到consumer处理的结果，继续生产，产生下一条消息

5、producer决定停止生产后，通过close关闭consumer，整个过程结束

""" 生产者——消费者模型(协程) """

def consumer():

r = ''

while True:

n = yield r

if not n:

return

print('[Consumer] Consuming %s' % n)

r = '200 OK'

def producer(c):

c.send(None)

n = 0

while n < 5:

n += 1

print('[Producer] Producing %s' % n)

r = c.send(n)

print('[Producer] Consumer return: %s' % r)

c.close()

cc = consumer()

producer(cc)

[Producer] Producing 1

[Consumer] Consuming 1

[Producer] Consumer return: 200 OK

[Producer] Producing 2

[Consumer] Consuming 2

[Producer] Consumer return: 200 OK

[Producer] Producing 3

[Consumer] Consuming 3

[Producer] Consumer return: 200 OK

[Producer] Producing 4

[Consumer] Consuming 4

[Producer] Consumer return: 200 OK

[Producer] Producing 5

[Consumer] Consuming 5

[Producer] Consumer return: 200 OK

Process finished with exit code 0

asyncio

asyncio是一个标准库，直接内置了对异步IO的支持

用asyncio实现HelloWorld

""" 用asyncio实现Hello World """

import asyncio

@asyncio.coroutine

def hello():

print('Hello World!')

# 异步调用asyncio.sleep

yield from asyncio.sleep(1)

print('Hello again!')

# 获取EventLoop

loop = asyncio.get_event_loop()

# 执行coroutine

loop.run_until_complete(hello())

loop.close()

Hello World!

Hello again!

Process finished with exit code 0

用asyncio的异步网络连接来获取请求的header

""" 用asyncio的异步网络连接来获取请求的header """

import asyncio

@asyncio.coroutine

def wget(host):

print('wget %s...' % host)

connect = asyncio.open_connection(host, 80)

reader, writer = yield from connect

header = 'GET / HTTP/1.0\r\nHost: %s\r\n\r\n' % host

writer.write(header.encode('utf-8'))

yield from writer.drain()

while True:

line = yield from reader.readline()

if line == b'\r\n':

break

print('%s header > %s' % (host, line.decode('utf-8').rstrip()))

writer.close()

loop = asyncio.get_event_loop()

tasks = [wget(host) for host in ['www.baidu.com', 'www.jd.com', 'www.qq.com', 'www.taobao.com']]

loop.run_until_complete(asyncio.wait(tasks))

loop.close()

wget www.qq.com...

wget www.baidu.com...

wget www.taobao.com...

wget www.jd.com...

www.qq.com header > HTTP/1.1 302 Moved Temporarily

www.qq.com header > Server: stgw/1.3.12.4_1.13.5

www.qq.com header > Date: Fri, 31 Jul 2020 08:08:14 GMT

www.qq.com header > Content-Type: text/html

www.qq.com header > Content-Length: 169

www.qq.com header > Connection: close

www.qq.com header > Location: https://www.qq.com/

www.baidu.com header > HTTP/1.0 200 OK

www.baidu.com header > Accept-Ranges: bytes

www.baidu.com header > Cache-Control: no-cache

www.baidu.com header > Content-Length: 14615

www.baidu.com header > Content-Type: text/html

www.baidu.com header > Date: Fri, 31 Jul 2020 08:08:14 GMT

www.baidu.com header > P3p: CP=" OTI DSP COR IVA OUR IND COM "

www.baidu.com header > P3p: CP=" OTI DSP COR IVA OUR IND COM "

www.baidu.com header > Pragma: no-cache

www.baidu.com header > Server: BWS/1.1

www.baidu.com header > Set-Cookie: BAIDUID=B5A4DF1D527248708FAA5CA3FFE230F9:FG=1; expires=Thu, 31-Dec-37 23:55:55 GMT; max-age=2147483647; path=/; domain=.baidu.com

www.baidu.com header > Set-Cookie: BIDUPSID=B5A4DF1D527248708FAA5CA3FFE230F9; expires=Thu, 31-Dec-37 23:55:55 GMT; max-age=2147483647; path=/; domain=.baidu.com

www.baidu.com header > Set-Cookie: PSTM=1596182894; expires=Thu, 31-Dec-37 23:55:55 GMT; max-age=2147483647; path=/; domain=.baidu.com

www.baidu.com header > Set-Cookie: BAIDUID=B5A4DF1D52724870C8528BC85E29577C:FG=1; max-age=31536000; expires=Sat, 31-Jul-21 08:08:14 GMT; domain=.baidu.com; path=/; version=1; comment=bd

www.baidu.com header > Traceid: 159618289405573632109291624733192979223

www.baidu.com header > Vary: Accept-Encoding

www.baidu.com header > X-Ua-Compatible: IE=Edge,chrome=1

www.taobao.com header > HTTP/1.1 301 Moved Permanently

www.taobao.com header > Server: Tengine

www.taobao.com header > Date: Fri, 31 Jul 2020 08:08:14 GMT

www.taobao.com header > Content-Type: text/html

www.taobao.com header > Content-Length: 278

www.taobao.com header > Connection: close

www.taobao.com header > Location: https://www.taobao.com/

www.taobao.com header > Via: cache4.cn806[,0]

www.taobao.com header > Timing-Allow-Origin: *

www.taobao.com header > EagleId: 968a619815961828940813509e

www.jd.com header > HTTP/1.1 302 Moved Temporarily

www.jd.com header > Server: nginx

www.jd.com header > Date: Fri, 31 Jul 2020 08:08:14 GMT

www.jd.com header > Content-Type: text/html

www.jd.com header > Content-Length: 138

www.jd.com header > Connection: close

www.jd.com header > Location: https://www.jd.com/

www.jd.com header > Access-Control-Allow-Origin: *

www.jd.com header > Timing-Allow-Origin: *

www.jd.com header > X-Trace: 302-1596182894095-0-0-0-0-0

www.jd.com header > Strict-Transport-Security: max-age=360

Process finished with exit code 0

为了更好的标识异步IO，我们可以使用async/await来替换@asyncio.coroutine和yield from

修改后的代码如下：

""" 用asyncio的异步网络连接来获取请求的header """

import asyncio

async def wget(host):

print('wget %s ...' % host)

connect = asyncio.open_connection(host, 80)

reader, writer = await connect

header = 'GET / HTTP/1.0\r\nHost: %s\r\n\r\n' % host

writer.write(header.encode('utf-8'))

await writer.drain()

while True:

line = await reader.readline()

if line == b'\r\n':

break

print('%s header > %s' % (host, line.decode('utf-8').rstrip()))

writer.close()

loop = asyncio.get_event_loop()

tasks = [wget(host) for host in ['www.qq.com', 'www.baidu.com', 'www.taobao.com']]

loop.run_until_complete(asyncio.wait(tasks))

loop.close()

蒟蒻写博客不易，加之本人水平有限，写作仓促，错误和不足之处在所难免，谨请读者和各位大佬们批评指正。

如需转载，请署名作者并附上原文链接，蒟蒻非常感激

名称：虐猫人薛定谔