python中asyncio的使用实例

Python 3通过asyncio模块以及await和async关键字提供了对异步I/O的支持。

步编程通常通过多任务协作处理的方式来实现，由于执行时间和顺序的不确定，因此需要通过钩子函数（回调函数）或者Future对象来获取任务执行的结果。

import asyncio

async def fetch(host):
    # 从指定站点抓取信息
    print(f'开始抓取{host}')
    # 和服务器建立连接
    reader, writer = await asyncio.open_connection(host, 80)
    # 构建请求行和请求头
    writer.write(b'GET / HTTP/1.1\r\n')
    writer.write(f'Host:{host}\r\n'.encode())
    writer.write(b'\r\n')
    # 清空缓存区
    await writer.drain()
    # 接收服务器的响应
    line = await reader.readline()
    while line != b'\r\n':
        print(line.decode().rsplit())
        line = await reader.readline()
    print('\n')
    writer.close()

def main():
    urls = ('www.baidu.com', 'www.csdn.net', 'www.163.com')
    # 获取系统默认事件循环
    loop = asyncio.get_event_loop()
    # 生成式语法，构造一个包含多个协议对象的列表
    tasks = [fetch(url) for url in urls]
    # wait： 将协程列表包装成Task(Future)并等待执行完成
    # run_until_complete： 运行任务，直到Future执行完成并返回结果
    loop.run_until_complete(asyncio.wait(tasks))
    loop.close()

if __name__ == '__main__':
    main()

执行结果：

开始抓取www.163.com
开始抓取www.baidu.com
开始抓取www.csdn.net
['HTTP/1.1', '200', 'OK']
['Date:', 'Tue,', '31', 'Mar', '2020', '15:08:46', 'GMT']
['Content-Type:', 'text/html;', 'charset=GBK']
['Transfer-Encoding:', 'chunked']
['Connection:', 'keep-alive']
['Expires:', 'Tue,', '31', 'Mar', '2020', '15:09:59', 'GMT']
['Server:', 'nginx']
['Cache-Control:', 'no-cache,no-store,private']
['Age:', '7']
['Vary:', 'Accept-Encoding']
['X-Ser:', 'BC80_dx-lt-yd-jiangsu-taizhou-4-cache-4,', 'BC84_dx-lt-yd-jiangsu-taizhou-4-cache-4,', 'BC233_ck-hubei-wuhan-2-cache-2,', 'BC196_ck-hubei-wuhan-2-cache-1']
['cdn-user-ip:', '49.211.105.136']
['cdn-ip:', '124.203.225.195']
['X-Cache-Remote:', 'HIT']
['cdn-source:', 'baishan']

['HTTP/1.1', '200', 'OK']
['Accept-Ranges:', 'bytes']
['Cache-Control:', 'no-cache']
['Connection:', 'keep-alive']
['Content-Length:', '14615']
['Content-Type:', 'text/html']
['Date:', 'Tue,', '31', 'Mar', '2020', '15:08:46', 'GMT']
['P3p:', 'CP="', 'OTI', 'DSP', 'COR', 'IVA', 'OUR', 'IND', 'COM', '"']
['P3p:', 'CP="', 'OTI', 'DSP', 'COR', 'IVA', 'OUR', 'IND', 'COM', '"']
['Pragma:', 'no-cache']
['Server:', 'BWS/1.1']
['Set-Cookie:', 'BAIDUID=9086941EE74E01736F33F9B8CF3844A2:FG=1;', 'expires=Thu,', '31-Dec-37', '23:55:55', 'GMT;', 'max-age=2147483647;', 'path=/;', 'domain=.baidu.com']
['Set-Cookie:', 'BIDUPSID=9086941EE74E01736F33F9B8CF3844A2;', 'expires=Thu,', '31-Dec-37', '23:55:55', 'GMT;', 'max-age=2147483647;', 'path=/;', 'domain=.baidu.com']
['Set-Cookie:', 'PSTM=1585667326;', 'expires=Thu,', '31-Dec-37', '23:55:55', 'GMT;', 'max-age=2147483647;', 'path=/;', 'domain=.baidu.com']
['Set-Cookie:', 'BAIDUID=9086941EE74E01739D5D416533CDA255:FG=1;', 'max-age=31536000;', 'expires=Wed,', '31-Mar-21', '15:08:46', 'GMT;', 'domain=.baidu.com;', 'path=/;', 'version=1;', 'comment=bd']
['Traceid:', '1585667326274850689010390091619431081419']
['Vary:', 'Accept-Encoding']
['X-Ua-Compatible:', 'IE=Edge,chrome=1']

['HTTP/1.1', '301', 'Moved', 'Permanently']
['Server:', 'openresty']
['Date:', 'Tue,', '31', 'Mar', '2020', '15:08:46', 'GMT']
['Content-Type:', 'text/html']
['Content-Length:', '182']
['Connection:', 'keep-alive']
['Keep-Alive:', 'timeout=20']
['Location:', 'https://www.csdn.net/']

Process finished with exit code 0