读书笔记 - python爬虫开发与项目实战 - 1.4 进程和线程

1.4 进程和线程

1.4.1 多进程

2. 使用multiprocessing模块创建多进程

import os
from multiprocessing import Process

# 子进程要执行的代码
def run_proc(name):
    print("Children process {} ({}) Running".format(name, os.getpid()))

if __name__ == "__main__":
    print("Parent process {}.".format(os.getpid()))
    
    for i in range(5):
        p = Process(target=run_proc, kwargs={
   "name": str(i)})
        print("Process will start.")
        p.start()
        
    p.join()
    print("Process end.")

Parent process 8072.
Process will start.
Process will start.
Process will start.
Process will start.
Process will start.
Process end.

%xmode Verbose
%pdb on
%debug

ERROR:root:No traceback has been produced, nothing to debug.


Automatic pdb calling has been turned ON

3. multiprocessing模块提供了一个Pool类来代表进程池对象

import os
import time
import random
from multiprocessing import Pool

def run_task(name):
    print("Task {} (pid = {}) is running...".format(name, os.getpid()))
    time.sleep(random.random() * 3)
    print("Task {} end.".format(os.getpid()))
    
if __name__ == "__main__":
    print("Current process {}.".format(os.getpid()))
    p = Pool(processes=3)