一、协程
1、又称微线程,纤程。英文名Coroutine.一句话说明什么是协程:协程是一种用户态的轻量级线程(相当于操作系统不知道它的存在,是用户控制的)。
2、协程拥有自己的寄存器上下文和栈(代码的必要的代码段和)。协程调度切换时,将寄存器上下文和栈保存到其他地方,在切回来的时候,恢复先前保存的寄存器上下文和栈,因此:协程能保留上一次调用时的状态(即所有局部状态的一个特定组合),每次过程重入时,就相当于进入上一次调用的状态,换种说法:进入上一次离开时所处逻辑流的位置。
3、协程一定是在单线程中运行的。
二、协程的优点与缺点
优点:
1、无需线程上下文切换的开销。
2、无需原子操作(最小级别的操作)锁定及同步的开销。
3、方便切换控制流,简化编程模型。
4、高并发+高扩展性+低成本:一个CPU支持上万的协程都不是问题,所以很适合用于高并发处理。
缺点:
1、无法利用多核资源:协程的本质是个单线程,它不能同时将单个CPU的多个核用上,协程需要和进程配合才能运行在多CPU上,当然我们日常所编写的绝大部分应用都没有这个必要,除非是cpu密集型应用。
2、进行阻塞(Blocking)操作(如IO时)会阻塞掉整个程序。
三、使用yield实现协程操作例子
1、使用yield实现的一个最简单的协程的效果
1 #!/usr/bin/python 2 # -*- coding : utf-8 -*- 3 # 作者: Presley 4 # 时间: 2018-12-4 5 # 邮箱:1209989516@qq.com 6 # 这是我用来练习python 协程的测试脚本 7 8 import time 9 import queue 10 11 def consumer(name): 12 print("starting eating baozi...") 13 while True: 14 new_baozi = yield 15 print("[%s] is eating baozi %s" %(name,new_baozi)) 16 17 def producer(): 18 r = con.__next__() 19 r = con2.__next__() 20 n = 0 21 while n < 5: 22 n += 1 23 con.send(n) 24 con2.send(n) 25 print("\033[32;1m[producer]\033[0m is making") 26 27 if __name__ == "__main__": 28 con = consumer("c1") 29 con2 = consumer("c2") 30 31 32 p = producer()
执行结果:
1 C:\Users\wohaoshuai\AppData\Local\Programs\Python\Python36\python.exe E:/PythonLearn/day16/pro_consume.py 2 starting eating baozi... 3 starting eating baozi... 4 [c1] is eating baozi 1 5 [c2] is eating baozi 1 6 [producer] is making 7 [c1] is eating baozi 2 8 [c2] is eating baozi 2 9 [producer] is making 10 [c1] is eating baozi 3 11 [c2] is eating baozi 3 12 [producer] is making 13 [c1] is eating baozi 4 14 [c2] is eating baozi 4 15 [producer] is making 16 [c1] is eating baozi 5 17 [c2] is eating baozi 5 18 [producer] is making 19 20 Process finished with exit code 0
2、greenlet
1 #!/usr/bin/python 2 # -*- coding : utf-8 -*- 3 # 作者: Presley 4 # 时间: 2018-12-4 5 # 邮箱:1209989516@qq.com 6 # 这是我用来练习python 协程的测试脚本 7 8 from greenlet import greenlet 9 10 def test1(): 11 print(12) 12 gr2.switch() 13 print(34) 14 gr2.switch() 15 16 def test2(): 17 print(56) 18 gr1.switch() 19 print(78) 20 21 gr1 = greenlet(test1) 22 gr2 = greenlet(test2) 23 gr1.switch()
执行结果:
1 C:\Users\wohaoshuai\AppData\Local\Programs\Python\Python36\python.exe E:/PythonLearn/day16/pro_consume.py 2 12 3 56 4 34 5 78 6 7 Process finished with exit code 0
3、gevent
a、gevent是一个第三方库,可以轻松通过gevent实现并发同步或异步编程,在gevent中用到的主要模式是greenlet,它是以C扩展模块形式接入Python的轻量级协程。greenlet全部运行在主程序操作系统进程的内部,但它们被协作式地调度。
1 #!/usr/bin/python 2 # -*- coding : utf-8 -*- 3 # 作者: Presley 4 # 时间: 2018-12-1 5 # 邮箱:1209989516@qq.com 6 # 这是我用来练习python 协程的测试脚本 7 8 import gevent 9 10 def foo(): 11 print("Running in foo") 12 gevent.sleep(1) 13 print("Explicit context switch to foo again") 14 15 def bar(): 16 print("Explicit context to bar") 17 gevent.sleep(1) 18 print("Implicit context switch back to bar") 19 20 def ex(): 21 print("Explicit context to ex") 22 gevent.sleep(1) 23 print("Implicit context switch back to ex") 24 25 gevent.joinall([ 26 gevent.spawn(foo), #类似产生一个协程的foo 27 gevent.spawn(bar), #产生一个协程的bar 28 gevent.spawn(ex) 29 ]) 30 31 #代码的效果为:第一个协程切换到第二个,第二个切换到第三个,然后又遇到sleep(模拟io)又切换到下一个,然后实现并发的协程的效果
执行结果
1 C:\Users\wohaoshuai\AppData\Local\Programs\Python\Python36\python.exe E:/PythonLearn/day16/pro_consume.py 2 Running in foo 3 Explicit context to bar 4 Explicit context to ex 5 Explicit context switch to foo again 6 Implicit context switch back to bar 7 Implicit context switch back to ex 8 9 Process finished with exit code 0
b、通过协程爬取网页实例
1 #!/usr/bin/python 2 # -*- coding : utf-8 -*- 3 # 作者: Presley 4 # 时间: 2018-12-5 5 # 邮箱:1209989516@qq.com 6 # 这是我用来练习python 协程的测试脚本 7 8 from gevent import monkey;monkey.patch_all() 9 import gevent 10 11 from urllib.request import urlopen 12 13 def f(url): 14 print("GET: %s" %url) 15 resp = urlopen(url) 16 data = resp.read() 17 print("%d bytes received from %s." %(len(data),url)) 18 19 gevent.joinall([ 20 gevent.spawn(f,"https://www.python.org/"), 21 gevent.spawn(f,"https://www.yahoo.com/"), 22 gevent.spawn(f,"https://github.com"), 23 ])
执行结果:
1 C:\Users\wohaoshuai\AppData\Local\Programs\Python\Python36\python.exe E:/PythonLearn/day16/pro_consume.py 2 GET: https://www.python.org/ 3 GET: https://www.yahoo.com/ 4 GET: https://github.com 5 80704 bytes received from https://github.com. 6 50008 bytes received from https://www.python.org/. 7 528149 bytes received from https://www.yahoo.com/. 8 9 Process finished with exit code 0
c、通过gevent实现单线程下的多socket并发
server端
1 #!/usr/bin/python 2 # -*- coding : utf-8 -*- 3 # 作者: Presley 4 # 时间: 2018-12-5 5 # 邮箱:1209989516@qq.com 6 # 这是我用来练习python 协程的测试脚本 7 8 9 import gevent 10 from gevent import socket,monkey 11 monkey.patch_all() #python中的一种黑魔法,只要写入一句话就自动的把python中的许多标准库变为非阻塞的模式 12 13 def server(port): 14 s = socket.socket() 15 s.bind(("0.0.0.0",port)) 16 s.listen(5000) 17 while True: 18 cli,addr = s.accept() 19 gevent.spawn(handle_request,cli) #执行handle_request函数,参数是cli,即客户端实例 20 def handle_request(s): 21 try: 22 while True: 23 data = s.recv(1024) #接收数据,这里设置成不阻塞 24 print("recv:",data) 25 s.send(data) 26 if not data: 27 s.shutdown(socket.SHUT_RD) #如果接收为空值,结束 28 except Exception as ex: 29 print(ex) 30 finally: 31 s.close() 32 33 if __name__ == "__main__": 34 server(8001)
client端
1 #!/usr/bin/python 2 # -*- coding : utf-8 -*- 3 # 作者: Presley 4 # 时间: 2018-12-5 5 # 邮箱:1209989516@qq.com 6 # 这是我用来练习python 协程的测试脚本 7 8 import socket 9 10 HOST = "localhost" 11 PORT = 8001 12 s = socket.socket() 13 s.connect((HOST,PORT)) 14 15 while True: 16 msg = input(">>:") 17 if not msg:continue 18 msg = msg.encode("utf-8") 19 s.sendall(msg) 20 data = s.recv(1024) 21 print("Received",data.decode("utf-8")) 22 s.close()