进程池

        当我们需要的进程数量不多的时候，我们可以使用multiprocessing的Process类来创建进程。但是如果我们需要的进程特别多的时候，手动创建工作量太大了，所以Python也为我们提供了Pool（池）的方式来创建大量进程。

1.进程池的创建

进程池的创建需要引入multiprocessing中的Pool，下面我们来创建一个简单的进程池来模拟多人同时下载的场景，假设进程池里有5个进程，有10个人同时下载，程序如下

from multiprocessing import Pool
import time


def download(path):
    print("The child processing is start download")
    print(path)
    print("Download completely")


if __name__ == '__main__':
    print("主进程开始了")
    # 申请进程池 若不传入参数 系统自动设定
    pool = Pool(5)
    for x in range(10):
        pool.apply_async(func=download, args=("www.baidu.com",))
    pool.close()
    pool.join()
    print("主进程结束了")

pool.close( )并不是停止进程，而是停止进程的申请的意思，有10 个人同时下载，而只有5个进程，注定要有5个人等待下载

pool.join( )阻断主进程，因为进程池开辟的进程是依赖于主进程的，主进程结束，进程池里的进程也结束。

进程池是为了避免出现 高并发时的进程被不断的创建和销毁 从而浪费系统资源的情况。

运行结果如下：

主进程开始了
The child processing is start download
www.baidu.com
Download completely
The child processing is start download
www.baidu.com
Download completely
The child processing is start download
www.baidu.com
Download completely
The child processing is start download
www.baidu.com
Download completely
The child processing is start download
www.baidu.com
Download completely
The child processing is start download
www.baidu.com
Download completely
The child processing is start download
www.baidu.com
Download completely
The child processing is start download
www.baidu.com
Download completely
The child processing is start download
www.baidu.com
Download completely
The child processing is start download
www.baidu.com
Download completely
主进程结束了

2. 进程池里的回调函数

from multiprocessing import Pool
import time


def download(path):
    print("The child processing is start download")
    print(path)
    print("Download completely")


# 回调函数
def save(fn):
    # fn是download的返回值
    print(fn)
    print("保存下载的文件")


if __name__ == '__main__':
    print("主进程开始了")
    # 申请进程池 若不传入参数 系统自动设定
    pool = Pool(5)
    for x in range(10):
        pool.apply_async(func=download, args=("www.baidu.com",), callback=save)
    # 停止申请
    pool.close()
    pool.join()
    print("主进程结束了")

运行结果如下：

主进程开始了
The child processing is start download
www.baidu.com
Download completely
None
保存下载的文件
The child processing is start download
www.baidu.com
Download completely
None
保存下载的文件
The child processing is start download
www.baidu.com
Download completely
None
保存下载的文件
The child processing is start download
www.baidu.com
Download completely
None
保存下载的文件
The child processing is start download
www.baidu.com
Download completely
None
The child processing is start download
保存下载的文件
www.baidu.com
Download completely
None
保存下载的文件
The child processing is start download
www.baidu.com
Download completely
None
保存下载的文件
The child processing is start download
www.baidu.com
Download completely
None
保存下载的文件
The child processing is start download
www.baidu.com
Download completely
None
保存下载的文件
The child processing is start download
www.baidu.com
Download completely
None
保存下载的文件
主进程结束了

3. Pool里的apply()和apply_async方法

apply( )方法调用时，进程池里的多个进程只有第一个运行完成后第二个才能运行

from multiprocessing import Pool
import time


def download(path):
    print("The child processing is start download")
    print(path)
    print("Download completely")
    time.sleep(1)


if __name__ == '__main__':
    print("主进程开始了")
    # 申请进程池 若不传入参数 系统自动设定
    pool = Pool(5)
    for x in range(10):
        pool.apply(func=download, args=("www.baidu.com",))
    # 停止申请
    pool.close()
    pool.join()
    print("主进程结束了")

apply_async，进程池里的多个进程同时运行

from multiprocessing import Pool
import time


def download(path):
    print("The child processing is start download")
    print(path)
    print("Download completely")
    time.sleep(1)


if __name__ == '__main__':
    print("主进程开始了")
    # 申请进程池 若不传入参数 系统自动设定
    pool = Pool(5)
    for x in range(10):
        pool.apply_async(func=download, args=("www.baidu.com",))
    # 停止申请
    pool.close()
    pool.join()
    print("主进程结束了")

它们两个的运行结果是截然不同的，有兴趣的可以自己试运行。