12.进程和线程
多任务的三种方式
- 多进程模式;
- 多线程模式;
- 多进程+多线程模式。
12.1多进程 multiprocessing
12.1.1fork
- Unix/Linux系统中,
fork()
函数调用一次在子进程返回0
,在父进程返回子进程ID。(父进程可fork出多个子进程,记下其ID,子进程调用getppid()
就可以拿到父进程ID
import os
print('Process (%s) start...' % os.getpid())
# Only works on Unix/Linux/Mac:
pid = os.fork()
if pid == 0:
print('I am child process (%s) and my parent is %s.' % (os.getpid(), os.getppid()))
else:
print('I (%s) just created a child process (%s).' % (os.getpid(), pid))
>>>
Process (876) start...
I (876) just created a child process (877).
I am child process (877) and my parent is 876.
- 有了
fork
调用,一个进程在接到新任务时就可以复制出一个子进程来处理新任务,常见的Apache服务器就是由父进程监听端口,每当有新的http请求时,就fork出子进程来处理新的http请求。
12.1.2multiprocessing
- 使用
multiprocessing
库的Process
类创建进程对象,start()
启动,join()
方法可以等待子进程结束后再继续往下运行,通常用于进程间的同步。
from multiprocessing import Process
import os
# 子进程要执行的代码
def run_proc(name):
print('Run child process %s (%s)...' % (name, os.getpid()))
if __name__=='__main__':
print('Parent process %s.' % os.getpid())
p = Process(target=run_proc, args=('test',))
print('Child process will start.')
p.start()
p.join()
print('Child process end.')
>>>
Parent process 928.
Child process will start.
Run child process test (929)...
Process end.
12.1.3Pool
- 如果要启动大量的进程,则可以用进程池批量创建子进程
from multiprocessing import Pool
import os, time, random
def long_time_task(name):
print('Run task %s (%s)...' % (name, os.getpid()))
start = time.time()
time.sleep(random.random() * 3)
end = time.time()
print('Task %s runs %0.2f seconds.' % (name, (end - start)))
if __name__=='__main__':
print('Parent process %s.' % os.getpid())
p = Pool(4) # 默认为CPU的核数
for i in range(5):
p.apply_async(long_time_task, args=(i,))
print('Waiting for all subprocesses done...')
p.close() # 先
p.join() # 后
print('All subprocesses don