Python标准模块--concurrent.futures
源码:https://docs.python.org/dev/library/concurrent.futures.html
#1 介绍:
concurrent.futures模块提供了高度封装的异步调用接口
ThreadPoolExecutor:线程池,提供异步调用
ProcessPoolExecutor: 进程池,提供异步调用
Both implement the same interface, which is defined by the abstract Executor class.两者都实现相同的接口
#2 基本方法:
submit(fn, *args, **kwargs)
异步提交任务
map(func, *iterables, timeout=None, chunksize=1)
取代for循环submit的操作
shutdown(wait=True)
相当于进程池的pool.close()+pool.join()操作
wait=True,等待池内所有任务执行完毕回收完资源后才继续
wait=False,立即返回,并不会等待池内的任务执行完毕
但不管wait参数为何值,整个程序都会等到所有任务执行完毕
submit和map必须在shutdown之前
result(timeout=None)
取得结果
add_done_callback(fn)
回调函数
有返回值得栗子:
import time
from concurrent.futures import ThreadPoolExecutor
def func(n):
time.sleep(2)
print(n)
return n*n
def call_back(m):
print('结果是 %s'%m.result()) # m拿到的是一个对象
tpool = ThreadPoolExecutor(5) # 默认 不要超过cpu个数*5
# tpool.map(func,range(20)) # 拿不到返回值
t_lst = []
for i in range(20):
t = tpool.submit(func,i)
t_lst.append(t)
tpool.shutdown() # close+join
print('主线程')
for t in t_lst:print('***',t.result())
回调函数栗子:
import time
from concurrent.futures import ThreadPoolExecutor
def func(n):
time.sleep(2)
print(n)
return n*n
def call_back(m):
print('结果是 %s'%m.result()) m拿到的是一个对象
tpool = ThreadPoolExecutor(5) 不要超过cpu个数*5
for i in range(20):
tpool.submit(func,i).add_done_callback(call_back)
ProcessPoolExecutor:栗子
from concurrent.futures import ThreadPoolExecutor,ProcessPoolExecutor
import os,time
def task(n):
print('%s is runing' %os.getpid())
time.sleep(1.5)
return n**2
if __name__ == '__main__':
executor=ProcessPoolExecutor(5)
futures=[]
for i in range(11):
future=executor.submit(task,i)
futures.append(future)
executor.shutdown(True)
print('+++>')
for future in futures:
print(future.result())