Python 多进程

在Windows中Python不能通过调用os模块中的fork()函数创建进程，但Python也提供了其他模块在Windows上也能实现

multiprocessing就是Python跨平台的多进程相关的模块

from multiprocessing impoty Process
import os
#os.getpid()获取当前进程的id

def run(name):
    print('Child process name = %s id = (%s)' % (name,os.getpid()))

if __name__ == '__main__':
    print('Parent process %s is runing...' % (os.getpid()))
    print('Child process will start run...')
    p = Process(target=fun,args=('I\'am child process',))
    p.start()
    p.join()
    print('Child process are destroy...')

join()方法可以等待子进程结束后再继续往下运行，通常用于进程间的同步。

输出

Parent process 7276...
Child process will start...
Run child process hello (3664)...
Child process is destroy...

Pool 如果要创建多个子进程，可以用进程池的方式创建多个子进程

from multiprocessing import Pool
import os,time,random

def long_time_request(name):
    print('Child process name = %s id = %s' % (name,os.getpid()))
    start = time.time()
    time.sleep(random.random()*10)
    end = time.time()
    print('%s child process use %0.2f seconds' % (name,end - start))

if __name__ == '__main__':
    print('Parent process id = %s is runing...' % (os.getpid()))
    print('Child process will start...')
    p = Pool(4)#进程池的大小默认是cpu(处理机)的个数
    for i in range(6):
        p.apply_async(long_time_request,args=(i,))
    p.close()#p调用join方法之前必须先调用close方法使进程池无法再添加新的进程
    p.join()#等待所有进程池中的进程执行完毕
    print('Child process are all done.')

输出

Parent process id = 7624 is runing...
Child process will start...
Child process name = 0 id = 5704
Child process name = 1 id = 6168
Child process name = 2 id = 4552
Child process name = 3 id = 8136
2 child process use 2.13 seconds
Child process name = 4 id = 4552
4 child process use 3.59 seconds
Child process name = 5 id = 4552
1 child process use 6.75 seconds
3 child process use 8.73 seconds
0 child process use 9.03 seconds
5 child process use 7.41 seconds
Child process are all done.

进程间通信

from multiprocessing import Process,Queue
import os,time,random

#写数据进程执行的代码
def write(q):
    print('Process %s to write' % (os.getpid()))
    for value in ['A','B','C']:
        print('Put %s to queue...' % value)
        q.put(value)
        time.sleep(random.random()*3)

#读数据的进程执行的代码
def read(q):
    print('Process %s to read' % (os.getpid()))
    while True:
        value = q.get(True)
        print('Get %s from queue.' % value)

if __name__ == '__main__':
    q = Queue()#注意这个队列是多进程模块中的
    pread = Process(target=read,args=(q,))
    pwrite = Process(target=write,args=(q,))
    pwrite.start()
    pread.start()
    pwrite.join()
    pread.terminate()#read进程中是死循环需强制终止

输出
Process 5364 to write
Put A to queue...
Process 7524 to read
Get A from queue.
Put B to queue...
Get B from queue.
Put C to queue...
Get C from queue.

在Unix/Linux下，multiprocessing模块封装了fork()调用，
使我们不需要关注fork()的细节。由于Windows没有fork调用
，因此，multiprocessing需要“模拟”出fork的效果，
父进程所有Python对象都必须通过pickle序列化再传到子进程去，
所有，如果multiprocessing在Windows下调用失败了，要先考虑是不是pickle失败了。