python多进程数据传递

最新推荐文章于 2024-03-11 18:50:50 发布

桂渊泉树

最新推荐文章于 2024-03-11 18:50:50 发布

阅读量529

点赞数

分类专栏： python

本文链接：https://blog.csdn.net/qq_30904887/article/details/102719139

版权

python 专栏收录该内容

5 篇文章 1 订阅

订阅专栏

python multiprocessing多进程数据传递使用列队太多bug了，有限制，
可以使用multiprocessing.Manager()来传递数据
可以解决queues列队的阻塞问题

import multiprocessing

def fun(q,number,lst):
    arr = []
    for i in lst:
        arr.append(i)
    for i in range(2):
        q.append(arr)

if __name__=='__main__':
    lst = list(range(200))
    thread_input_list = dict()
    partition_point = int(len(lst)/4)
    for thread_number in range(0,4):
        thread_input_list[thread_number] = lst[(thread_number*partition_point):(thread_number+1)*partition_point]
      
    thread_position_list = dict()
    processes = dict()
    queues = dict()
    with multiprocessing.Manager() as m:
        for thread in range(0,4):
            queues[thread] = m.list()
            processes[thread] = multiprocessing.Process(target=fun,args=(queues[thread],thread,thread_input_list[thread]))
            processes[thread].start()
        for thread in range(0,4):
            processes[thread].join()
            thread_position_list[thread] = queues[thread]
        lst = []
        for key,value in thread_position_list.items():
            lst.extend(value)
    print(lst)