Python3 协程、线程、进程example & 对比

概述

除了一般编程语言都具备的线程、进程，python还有一种‘轻量级’的线程——协程。

进程

不必赘述，和其他语言类似。

线程

python的线程是一个奇葩的存在，不同于其他语言。随着硬件技术的高速发展，各CPU厂商在核心频率上的比赛已经被多核所取代。为了更好的利用多核优势，随之衍生出了多线程编程，python自然也不能免俗。但是多线程带来的问题是线程间数据一致性和状态同步，最简单的就是加一把锁，于是产生了一把名叫GIL的超级锁。这把锁的存在，使得python的多线程事实上依然只是单线程，因为GIL限制了同一时刻，只能有一个线程拿到锁。正因为GIL的存在，导致了python的多线程不是并行，只能算并发，性能极低。

协程

协程是一种轻量级的线程。与线程的区别是，线程是由操作系统调度切换，而协程有python程序自己控制调度。

例程

多进程

关键词：进程池、进程间通信、信号捕捉

#!/usr/bin/python3

import os
import signal
import time
from multiprocessing import Pool, Queue


# 新建一个queue用于进程间通信
chan = Queue()


def producer():
    while 1:
        now = time.time()
        print('producer time: {}'.format(now))
        # 入队
        chan.put_nowait(now)
        time.sleep(1)


def consumer():
    while 1:
        if not chan.empty():
            # 出队
            now = chan.get_nowait()
            print('consumer time: {}'.format(now))


def sig_handler(signum, frame):
    pid = os.getpid()
    ppid = os.getppid()
    print('[{}][{}]signal {} received, program exit...'.format(pid, ppid, signum))
    exit(0)


if __name__ == '__main__':
    signal.signal(signal.SIGINT, sig_handler)
    signal.signal(signal.SIGHUP, sig_handler)
    signal.signal(signal.SIGTERM, sig_handler)
    # 新建可容纳两个子进程得进程池
    pools = Pool(processes=2)
    try:
        pools.apply_async(producer, args=())
        pools.apply_async(consumer, args=())
    except Exception as e:
        print('exception {} capture, program exit...'.format(e))
        exit(1)

    while 1:
        pass

多线程

关键词：多线程、线程间通信、信号捕捉

#!/usr/bin/python3

import os
import signal
import time
import _thread
import queue


# 新建一个queue用于线程间通信
chan = queue.Queue()


def producer():
    while 1:
        now = time.time()
        print('producer time: {}'.format(now))
        # 入队
        chan.put_nowait(now)
        time.sleep(1)


def consumer():
    while 1:
        if not chan.empty():
            # 出队
            now = chan.get_nowait()
            print('consumer time: {}'.format(now))


def sig_handler(signum, frame):
    pid = os.getpid()
    ppid = os.getppid()
    print('[{}][{}]signal {} received, program exit...'.format(pid, ppid, signum))
    exit(0)


if __name__ == '__main__':
    signal.signal(signal.SIGINT, sig_handler)
    signal.signal(signal.SIGHUP, sig_handler)
    signal.signal(signal.SIGTERM, sig_handler)

    try:
        _thread.start_new_thread(producer, ())
        _thread.start_new_thread(consumer, ())
    except:
        print('Error: unable to start thread')

    while 1:
        pass

多协程

关键词：协程、协程间通信、协程调度
在编写协程时需要注意，协程需要程序自己控制切换，也就是代码中调用await asyncio.sleep(1)，本质是让producer协程陷入睡眠，交出CPU使用权，然后consumer获得执行时机，轮换交复。

#!/usr/bin/python3

import os
import signal
import time
import asyncio


# 新建一个queue用于协程间通信
chan = asyncio.Queue()


async def producer():
    while 1:
        now = time.time()
        # 入队
        await chan.put(now)
        print('producer time: {}'.format(now))
        await asyncio.sleep(1)


async def consumer():
    while 1:
        # 出队
        now = await chan.get()
        print('consumer time: {}'.format(now))
        await asyncio.sleep(1)


def sig_handler(signum, frame):
    pid = os.getpid()
    ppid = os.getppid()
    print('[{}][{}]signal {} received, program exit...'.format(pid, ppid, signum))
    exit(0)


if __name__ == '__main__':
    signal.signal(signal.SIGINT, sig_handler)
    signal.signal(signal.SIGHUP, sig_handler)
    signal.signal(signal.SIGTERM, sig_handler)

    tasks = [asyncio.ensure_future(producer()),
             asyncio.ensure_future(consumer())]
    loop = asyncio.get_event_loop()
    try:
        loop.run_until_complete(asyncio.wait(tasks))
    except Exception as e:
        for task in tasks:
            task.cancel()
        loop.stop()
        loop.run_forever()
    finally:
        loop.close()