python concurrent 多进程、多线程传递多参数的真正方法

 concurrent.futures模块可以使用多进程、多线程，concurrent.futures.Executor.map(fn, *iterables)可以简洁高效地并发执行函数，第一个参数为函数名，第二个参数为函数的参数。但网上的教程大多只传递给函数一个参数，实用性太低。

网上搜到的给map()函数传递多参数的方法有两种，使用zip()函数打包或使用偏函数，都是错的！！！

实际上，给map()函数传递多参数，依次传递就行了。下面是例子。

from concurrent.futures import ProcessPoolExecutor


def test(data1,data2,k1=1,k2=10,k3=-5):
    print(f'${data1}_${data2}_${k1}_${k2}_${k3}!')

def main():
    datalist_1 = [1,2,3,4,5,6]
    datalist_2 = ['a','b','c','d','e','f']
    k1 = [1,3,5,7,9,11]
    k2 = [2,4,6,8,10,12]
    k3 = [-1,-3,-5,-7,-9,-11]
    with ProcessPoolExecutor() as executor:
        result = executor.map(test, datalist_1,datalist_2,k1,k2,k3)

if __name__ == '__main__':
    main()
        

在这个例子中，有默认参数，如果部分默认参数不变，最简单的办法是把默认参数再传递一遍，结果也是一样的。

def main_1():
    datalist_1 = [1,2,3,4,5,6]
    datalist_2 = ['a','b','c','d','e','f']
    k1 = [1,3,5,7,9,11]
    k2 = [10] * len(datalist_1)
    k3 = [-5] * len(datalist_1)
    with ProcessPoolExecutor() as executor:
        result = executor.map(test, datalist_1,datalist_2,k1,k2,k3)

如果函数的参数不固定，有可变参数，不好直接传递，可以采用submit的形式。

from concurrent.futures import as_completed

def main_2():
    datalist_1 = [1,2,3,4,5,6]
    datalist_2 = ['a','b','c','d','e','f']
    k1 = [{'k1':1},{'k1':3},{'k1':5},{'k1':7},{'k1':9},{'k1':11}]
    k3 = [-1,-3,-5,-7,-9,-11]
    with ProcessPoolExecutor() as executor:
        jobs, result = [], []
        for i in range(len(datalist_1)):
            jobs.append(executor.submit(test, datalist_1[i], datalist_2[i], **k1[i], **{'k3':k3[i]}))
        for job in as_completed(jobs):
            result.append(job.result())

多线程的话是一样的，把ProcessPoolExecutor换成ThreadPoolExecutor就行了。