python multiprocessing：多进程与进程池

创建新进程

使用 multiprocessing 模块中的 Process 类创建进程：

from multiprocessing import Process

创建进程与创建线程的语法十分相似。
多进程的问题： 进程之间不能共享资源，例如下面的例子，如果process2 是 ask_user 的进程，则会出现错误 EOFError: EOF when reading a line

process2 = Process(target=ask_user) # Will cause EOFError!!

因为 process2 无法从终端读取数据。

import time
from multiprocessing import Process

def ask_user():
    start = time.time()
    user_input = input("Enter your name: ")
    greet = f"Hello, {user_input}"
    print(greet)
    print(f'ask_user, {time.time() - start}')

def complex_calculation():
    start = time.time()
    print('Started calculating...')
    [x**2 for x in range(20000000)]
    print(f'complex_calculation, {time.time() - start}')

# for windows: put my_process.start(), my_process.join() 
# in this if statement is important
# or new processes will continuously start
if __name__ == '__main__':
    start = time.time()
    ask_user()
    complex_calculation()
    print(f'Single thread total time: {time.time() - start}')

    process = Process(target=complex_calculation)

	# if here target = ask_user, input cannot be read
	#  user_input = input("Enter your name: ")
	# EOFError: EOF when reading a line
	# because resources cannot be shared among processes
    process2 = Process(target=complex_calculation)

    process.start()
    process2.start()

    start = time.time()

    process.join()
    process2.join()

    print(f'Two processes total time: {time.time() - start}')

进程池

process pool 与 thread pool 的语法十分相似：

import time
from concurrent.futures import ProcessPoolExecutor

def ask_user():
    start = time.time()
    user_input = input("Enter your name: ")
    greet = f"Hello, {user_input}"
    print(greet)
    print(f'ask_user, {time.time() - start}')


def complex_calculation():
    start = time.time()
    print('Started calculating...')
    [x**2 for x in range(20000000)]
    print(f'complex_calculation, {time.time() - start}')


if __name__ == '__main__':
    start = time.time()
    complex_calculation()
    complex_calculation()
    print(f'Single thread total time: {time.time() - start}')

    start = time.time()
    with ProcessPoolExecutor(max_workers=2) as pool:
        pool.submit(complex_calculation)
        pool.submit(complex_calculation)
    print(f'Two processes total time: {time.time() - start}')

输出：

Started calculating...
complex_calculation, 3.7751059532165527
Started calculating...
complex_calculation, 3.809253215789795
Single thread total time: 7.584359169006348
Started calculating...
Started calculating...
complex_calculation, 3.9134361743927
complex_calculation, 3.94466495513916
Two processes total time: 4.0857954025268555

进程池的写法比创建进程简洁。

使用进程，程序运行时间明显缩短。两次调用 complex_calculation()， 如果不使用两个进程，调用时间累加，需要 7.5 秒，使用两个进程单独完成计算，则所需时间缩短为 4 秒。

一般情况下，如果是有需要等待的多任务程序，使用多线程。

如果 CPU 多核，并且需要执行多项使用CPU的复杂计算，使用多进程。