Python多进程操作初探

fock()

Unix/Linux操作系统提供了一个fork()系统调用,Python的os模块封装了常见的系统调用，其中就包括fork，可以在Python程序中轻松创建子进程：

import os

print('Process (%s) start...' % os.getpid())
# Only works on Unix/Linux/Mac:
pid = os.fork()
if pid == 0:
    print('I am child process (%s) and my parent is %s.' % (os.getpid(), os.getppid()))
else:
    print('I (%s) just created a child process (%s).' % (os.getpid(), pid))

运行结果：

Process (37) start...
I (37) just created a child process (38).
Process (37) start...
I am child process (38) and my parent is 37.

Windows没有fork调用，上面的代码在Windows上无法运行。

multiprocessing

multiprocessing模块提供了一个Process类来代表一个进程对象

from multiprocessing import Process
import os

# 子进程要执行的代码
def run_proc(name):
    print('Run child process %s (%s)...' % (name, os.getpid()))

if __name__=='__main__':
    print('Parent process %s.' % os.getpid())
    p = Process(target=run_proc, args=('test',))
    print('Child process will start.')
    p.start()
    p.join()
    print('Child process end.')

运行结果：

D:\develop\python3.6\python.exe D:/develop/pythondemo/123/demo.py
Parent process 13264.
Child process will start.
Run child process test (10648)...
Child process end.

创建子进程时，只需要传入一个执行函数和函数的参数，创建一个Process实例，用start()方法启动，这样创建进程比fork()还要简单。

join()方法可以等待子进程结束后再继续往下运行，通常用于进程间的同步。

Pool

from multiprocessing import Pool
import os, time, random

def long_time_task(name):
    print('Run task %s (%s)...' % (name, os.getpid()))
    start = time.time()
    time.sleep(random.random() * 3)
    end = time.time()
    print('Task %s runs %0.2f seconds.' % (name, (end - start)))

if __name__=='__main__':
    print('Parent process %s.' % os.getpid())
    p = Pool(4)
    for i in range(5):
        p.apply_async(long_time_task, args=(i,))
    print('Waiting for all subprocesses done...')
    p.close()
    p.join()
    print('All subprocesses done.')

运行结果：

D:\develop\python3.6\python.exe D:/develop/pythondemo/123/demo.py
Parent process 11476.
Waiting for all subprocesses done...
Run task 0 (11512)...
Run task 1 (12492)...
Run task 2 (8628)...
Run task 3 (4300)...
Task 2 runs 0.22 seconds.
Run task 4 (8628)...
Task 0 runs 0.66 seconds.
Task 4 runs 1.03 seconds.
Task 3 runs 1.76 seconds.
Task 1 runs 2.89 seconds.
All subprocesses done.