理解Python的PoolExecutor

Demo代码和引用知识点都参考自《理解Python并发编程一篇就够了|PoolExecutor篇》–董伟明或作者个人公众号Python之美， 《Python Cookbook》和Python并发编程之线程池/进程池。

ThreadPoolExecutor 和ProcessPoolExecutor分别对threading和multiprocessing进行了高级抽象，暴露出简单的统一接口。

通过ProcessPoolExecutor 来做示例。

主要提供两个方法map() 和submit()。

map() 方法主要用来针对简化执行相同的方法，如下例：

# -*- coding: utf-8 -*-

from concurrent.futures import ProcessPoolExecutor

def fib(n, test_arg):
    if n > 30:
        raise Exception('can not > 30, now %s' % n)
    if n <= 2:
        return 1
    return fib(n-1, test_arg) + fib(n-2, test_arg)

def use_map():
    nums = [random.randint(0, 33) for _ in range(0, 10)]
    with ProcessPoolExecutor() as executor:
        try:
            results = executor.map(fib, nums, nums)
            for num, result in zip(nums, results):
                print('fib(%s) result is %s.' % (num, result))
        except Exception as e:
            print(e)

执行上例，输出如下，当num为30时抛出异常捕获后程序停止运行。

...
fib(19) result is 4181.
fib(11) result is 89.
fib(2) result is 1.
fib(5) result is 5.
fib(24) result is 46368.
fib(2) result is 1.
can not > 30, now 33

---

使用submit()方法。

# -*- coding: utf-8 -*-

from concurrent.futures import ProcessPoolExecutor, as_completed
import random

def fib(n, test_arg):
    if n > 30:
        raise Exception('can not > 30, now %s' % n)
    if n <= 2:
        return 1