Python3之concurrent.futures实现线程池，进程池

关于concurrent.futures的官方文档

concurrent.futures

进程池

代码演示

from concurrent.futures import ProcessPoolExecutor, ThreadPoolExecutor


def deal_num(n):
    return n ** 2


if __name__ == '__main__':
    # 参数默认是CPU个数
    p = ProcessPoolExecutor()

    obj_l = []
    for i in range(10):
        # 异步提交任务
        obj = p.submit(deal_num, i)
        obj_l.append(obj)

    p.shutdown()  # 等同于p.close(),p.join()
    print([obj.result() for obj in obj_l])

代码解读

1. An Executor subclass that executes calls asynchronously using a pool of at most max_workers processes. If max_workers is None or not given, it will default to the number of processors on the machine. If max_workers is lower or equal to 0, then a ValueError will be raised.

p = ProcessPoolExecutor()

1. 如果没有p.shutdown()，主进程也会等子进程执行完，但是先将主进程代码之心完毕才在执行子进程。

线程池

代码演示

from concurrent.futures import ProcessPoolExecutor, ThreadPoolExecutor


def deal_num(n):
    return n ** 2


if __name__ == '__main__':
    # 参数默认是CPU个数*5
    p = ThreadPoolExecutor()

    obj_l = []
    for i in range(10):
        # 异步提交任务
        obj = p.submit(deal_num, i)
        obj_l.append(obj)

    p.shutdown()  # 等同于p.close(),p.join()
    print([obj.result() for obj in obj_l])

代码解读

Changed in version 3.5: If max_workers is None or not given, it will default to the number of processors on the machine, multiplied by 5.

p = ThreadPoolExecutor()