Python基础（18）——并行编程

一、概述

1.非并发

a.程序由单个步骤序列构成
b.包含独立子任务的程序执行性能低

2.并发

a.异步、高效
b.分解子任务、简化流程与逻辑

3.进程 process

a.一个程序的执行实例
b.每个进程有自己的地址空间、内存、数据栈及辅助数据

4.线程thread

同一进程内，可被并行激活的控制流
共享相同上下文（空间地址、数据结构）
特点：便于信息共享和通信，线程访问顺序差异会导致结果不一致（条件 race condition）

5.Python GIL 全局解释器锁

Global Interpreter Lock
Python 代码由虚拟机（解释器主循环）控制
主循环同时只能有一个控制线程执行

二、多线程

1.引入单线程实例

2._thread 模块

（前面加了下划线（—） python意不建议使用）
特点：没有控制进程结束的机制，只有一个同步原语（锁），功能少于threading模块
开始：_thread.start_new_thread(function,arg,**kwargs=None)

3. threading 模块

a. .Thread 线程类
构造：.Thread(target = 目标函数,args = (参数,))
自定义Thread派生类，重写run方法逻辑

.start() 启动线程
.join() 要求主线程等待
.name 线程名称
b. threading.current_thread() 获取当前线程

import threading
import time
"""threading.thread()实现多线程
"""
def worker(n):
    print('函数执行开始于:{}'.format(time.ctime()))
    time.sleep(n)
    print(f'函数执行结束于:{time.ctime()}')

def main():
    print('【主函数执行开始于:{}】'.format(time.ctime()))
    # _thread.start_new_thread(worker, (4,))
    # _thread.start_new_thread(worker, (2,))
    threads = []
    t1 = threading.Thread(target=worker, args=(4,))
    threads.append(t1)
    t2 = threading.Thread(target=worker, args=(2,))
    threads.append(t2)

    for t in threads:
        t.start()
    for t in threads:
        t.join()
    # time.sleep(4)
    print(f'【主函数执行结束于:{time.ctime()}】')

if __name__ == '__main__':
    main()

查看线程

import threading
import time
"""threading.thread()实现多线程
"""
def worker(n):
    print('{} 函数执行开始于:{}'.format(threading.current_thread().name,time.ctime()))
    time.sleep(n)
    print(f'{threading.current_thread().name}函数执行结束于:{time.ctime()}')

def main():
    print('【主函数执行开始于:{}】'.format(time.ctime()))
    # _thread.start_new_thread(worker, (4,))
    # _thread.start_new_thread(worker, (2,))
    threads = []
    t1 = threading.Thread(target=worker, args=(4,))
    threads.append(t1)
    t2 = threading.Thread(target=worker, args=(2,))
    threads.append(t2)

    for t in threads:
        t.start()
    for t in threads:
        t.join()
    # time.sleep(4)
    print(f'【主函数执行结束于:{time.ctime()}】')

if __name__ == '__main__':
    main()

构造自定义thread派生类结果·同上

import threading
import time
"""threading.thread()实现多线程
"""
def worker(n):
    print('{} 函数执行开始于:{}'.format(threading.current_thread().name,time.ctime()))
    time.sleep(n)
    print(f'{threading.current_thread().name}函数执行结束于:{time.ctime()}')

class MyThread(threading.Thread): #自定义Thread派生类，继承Thread基类
    def __init__(self, func, args):
        threading.Thread.__init__(self)
        self.func = func
        self.args = args

    def run(self):
        self.func(*self.args)

def main():
    print('【主函数执行开始于:{}】'.format(time.ctime()))
    # _thread.start_new_thread(worker, (4,))
    # _thread.start_new_thread(worker, (2,))
    threads = []
    # t1 = threading.Thread(target=worker, args=(4,))
    t1 = MyThread(worker, (4,))
    threads.append(t1)

    # t2 = threading.Thread(target=worker, args=(2,))
    t2 = MyThread(worker, (2,))
    threads.append(t2)

    for t in threads:
        t.start()
    for t in threads:
        t.join()
    # time.sleep(4)
    print(f'【主函数执行结束于:{time.ctime()}】')

if __name__ == '__main__':
    main()

c. threading.Lock 同步原语：锁
.acquire() #锁住
.release() #释放
支持上下文操作 with lock:


运行结果

加锁后按照顺序

运行·结果·

三、queue模块

（1）Queue FIFO

.Queue(maxsize=0)构造实例
.put(item, block = True, timeout =None) 放入数据项
.get(blcok = True, timeout = None) 获取数据项
.task_done() 声明当前队列任务处理完毕
.join() 队列所有项处理完毕前阻塞
eg:



运行结果：

（2）LifeQueue LIFO

（3）PriorityQueue 优先队列

四、multprocessing 模块

import multiprocessing
import time
"""
多进程实例
"""
def func(n):
    print(f'{multiprocessing.current_process().name} 执行开始于:{time.ctime()}')
    time.sleep(n)
    print(f'{multiprocessing.current_process().name} 执行结束于:{time.ctime()}')

def main():
    print(f'主函数运行于：{time.ctime()}')
    processes = []
    p1 = multiprocessing.Process(target=func, args=(4,))
    processes.append(p1)

    p2 = multiprocessing.Process(target=func, args=(2,))
    processes.append(p2)
    print(f'主函数结束于：{time.ctime()}')

    for p in processes:
        p.start()

    for p in processes:
        p.join()

    print(f'主函数结束于：{time.ctime()}')

if __name__ == '__main__':
    main()

E:\Python3.7\python.exe E:/Pycharm/untitled3/multprocess.py
主函数运行于：Fri Jan 31 15:05:19 2020
主函数结束于：Fri Jan 31 15:05:19 2020
Process-1 执行开始于:Fri Jan 31 15:05:19 2020
Process-2 执行开始于:Fri Jan 31 15:05:19 2020
Process-2 执行结束于:Fri Jan 31 15:05:21 2020
Process-1 执行结束于:Fri Jan 31 15:05:23 2020
主函数结束于：Fri Jan 31 15:05:23 2020

Process finished with exit code 0

五、concurrent.futures 模块

(1) ThreadPoolExecutor
(2) ProcessPoolExecutor

import time
import concurrent.futures
from time import perf_counter
"""
concurrent_futures应用
"""
numbers = list(range(1,11))

def count(n):
    for i in range(100000):
        i += i
    return i*n

def worker(x):
    result = count(x)
    print(f'数字:{x}的计算结果是:{result}')

#顺序执行
def sequential_execution():
    start_time = perf_counter()
    for i in numbers:
        worker(i)
    print(f'顺序执行花费时间:{perf_counter()-start_time}秒')

#线程池执行
def threading_execution():
    start_time = perf_counter()
    with concurrent.futures.ThreadPoolExecutor(max_workers=5) as executor:
        for i in numbers:
            executor.submit(worker, i)
    print(f'线程池执行花费时间:{perf_counter() - start_time}秒')

#进程池执行
def process_execution():
    start_time = perf_counter()
    with concurrent.futures.ProcessPoolExecutor(max_workers=5) as executor:
        for i in numbers:
            executor.submit(worker, i)
    print(f'进程池执行花费时间:{perf_counter() - start_time}秒')

if __name__ == '__main__':
    # sequential_execution()
     threading_execution()
    # process_execution()