python多进程相关问题

ProcessPoolExecutor

在进程池中再开进程池

from concurrent.futures import ProcessPoolExecutor

def work_a(b):
    print(f"{b} in work_a")
    with ProcessPoolExecutor() as pool2:
        for a in b:
            pool2.submit(work_b, a)

def work_b(b):
    print(f"{b} in work_b\n")

with ProcessPoolExecutor() as pool:
    a = pool.submit(work_a, "abnaoiwenfionawioef")
    a.result()

不过先不要用with，先创建一个ProcessPoolExecutor的对象

搜一下ProcessPoolExecutor 加上python关键字可以搜到这个的文档

单纯的开进程池，代码示例

from concurrent.futures import ProcessPoolExecutor

with ProcessPoolExecutor(max_workers=WORKERS) as parallel_pool:
    for i in range(WORKERS):
        temp = [bbox_tuple for j, bbox_tuple in enumerate(bbox_list) if j % WORKERS == i]
        p = parallel_pool.submit(filt_bbox, layer, temp, border_entity_info, drawing_type, crop_entity_image, entity_save_path)
        parallel_list.append(p)

    for p in as_completed(parallel_list):
        result = p.result()
        border_entity_bbox_dict[layer].extend(result)

with ProcessPoolExecutor(max_workers=PROCESS_WORKER_NUM) as executor:
    futures = [executor.submit(get_door_base_coords_by_entity,
                               entity_lists[num*i//PROCESS_WORKER_NUM:num*(i+1)//PROCESS_WORKER_NUM],
                               border_entity_info, arc_entity_dict, line_entity_dict, ellipse_entity_dict,
                               w_range, h_range, wall_pillar_line_dict, line_dash_entity_list, line_entity_list)
               for i in range(PROCESS_WORKER_NUM)]
    for f in as_completed(futures):
        try:
            dw_line_list, used_origin_line_list = f.result()
            for line in dw_line_list:
                if line[3] == 'door_base_coords':
                    door_base_coords.append(line[0])
                cv2.line(img_space_copy, tuple(line[0][:2]), tuple(line[0][2:4]), line[1], line[2])
            used_line_list.extend(used_origin_line_list)
        except BaseException as e:
            print('ProcessPool break down:', e)

multiprocessing

参考文档：Python的进程库multiprocessing中，pool组件的使用(apply,apply_async,map,map_async)_西门大盗的博客-CSDN博客

import multiprocessing
import time

def get(seconds):
    time.sleep(seconds)
    print("process name: " + multiprocessing.current_process().name)

if __name__ == "__main__":
    start = time.time()
    pool = multiprocessing.Pool(5)
    for item in range(10):
        pool.apply_async(get, args=(1,))  # apply_async方法，通过args参数传参，进程之前不阻塞，真正意义上的并行

    pool.close()  # close方法关闭进程池，此时不允许新的子进程进入。close方法必须在join方法调用前调用
    pool.join()  # 主进程被阻塞，等子进程全部执行完毕后再执行print
    print("test end", time.time() - start)

进程间、线程间通信 - Queue的讲解

Python Queue模块详解_Python_脚本之家

Python 队列（Queue）用法_Survivior_Y的博客-CSDN博客_python queue

进程、进程池（两种实现方法）、进程通信（两种方法）等的详细讲解

Python 多进程和进程池 - 他山之石·玉 - 博客园

多进程/进程池/进程间共享数据的踩坑记录

python多进程/进程池/进程间共享数据实用场景分析和实践踩坑记录_苹果姐的博客-CSDN博客

multiprocessing.Queue 讲解

Python进程间通信 multiProcessing Queue队列实现详解 - 开发技术 - 亿速云