python之多进程

1. multiprocessing

   multiprocessing模块就是跨平台版本的多进程模块。

   from multiprocessing import Process
   import os
   
   # 子进程要执行的代码
   def run_proc(name):
       print('Run child process %s (%s)...' % (name, os.getpid()))
   
   if __name__=='__main__':
       print('Parent process %s.' % os.getpid())
       p = Process(target=run_proc, args=('test',))
       print('Child process will start.')
       p.start()
       p.join()#join()方法可以等待子进程运行结束，再继续往下运行
       print('Child process end.')
   

   执行结果

   Parent process 928.
   Child process will start.#start()方法让子进程启动
   Run child process test (929)...
   Child Process end.
   

2. 进程池Pool

   我们用apply_async(def,args=(元组))方法创建进程
   该方法表示异步非阻塞：即进程池中的任何一个进程结束了都可以开始下一个进程

   from multiprocessing import Pool
   import os, time, random
   
   def long_time_task(name):
       print('Run task %s (%s)...' % (name, os.getpid()))
       start = time.time()
       time.sleep(random.random() * 3)
       end = time.time()
       print('Task %s runs %0.2f seconds.' % (name, (end - start)))
   
   if __name__=='__main__':
       print('Parent process %s.' % os.getpid())
       p = Pool(4)
       for i in range(5):
           p.apply_async(long_time_task, args=(i,))
       print('Waiting for all subprocesses done...')
       p.close()#表示停止添加新的process
       p.join()
       print('All subprocesses done.')
   

   执行结果：

   Parent process 669.
   Waiting for all subprocesses done...
   Run task 0 (671)...
   Run task 1 (672)...
   Run task 2 (673)...
   Run task 3 (674)...
   Task 2 runs 0.14 seconds.
   Run task 4 (673)...
   Task 1 runs 0.27 seconds.
   Task 3 runs 0.86 seconds.
   Task 0 runs 1.41 seconds.
   Task 4 runs 1.91 seconds.
   All subprocesses done.
   

   这里task4没有在task3后面直接执行是因为初始化Pool(4)给p，所以只能等待进程池中的四个进程中的一个执行完，才能执行task4