Demo代码和引用知识点都参考自《理解Python并发编程一篇就够了|PoolExecutor篇》–董伟明或作者个人公众号Python之美, 《Python Cookbook》和Python并发编程之线程池/进程池。
ThreadPoolExecutor
和ProcessPoolExecutor
分别对threading
和multiprocessing
进行了高级抽象,暴露出简单的统一接口。
通过ProcessPoolExecutor
来做示例。
主要提供两个方法map()
和submit()
。
map()
方法主要用来针对简化执行相同的方法,如下例:
# -*- coding: utf-8 -*-
from concurrent.futures import ProcessPoolExecutor
def fib(n, test_arg):
if n > 30:
raise Exception('can not > 30, now %s' % n)
if n <= 2:
return 1
return fib(n-1, test_arg) + fib(n-2, test_arg)
def use_map():
nums = [random.randint(0, 33) for _ in range(0, 10)]
with ProcessPoolExecutor() as executor:
try:
results = executor.map(fib, nums, nums)
for num, result in zip(nums, results):
print('fib(%s) result is %s.' % (num, result))
except Exception as e:
print(e)
执行上例,输出如下,当num
为30时抛出异常捕获后程序停止运行。
...
fib(19) result is 4181.
fib(11) result is 89.
fib(2) result is 1.
fib(5) result is 5.
fib(24) result is 46368.
fib(2) result is 1.
can not > 30, now 33
使用submit()
方法。
# -*- coding: utf-8 -*-
from concurrent.futures import ProcessPoolExecutor, as_completed
import random
def fib(n, test_arg):
if n > 30:
raise Exception('can not > 30, now %s' % n)
if n <= 2:
return 1