并发编程-线程

Jiucheng18

已于 2023-03-02 21:33:05 修改

阅读量434

点赞数 1

分类专栏： # python高阶文章标签： python 开发语言

于 2023-03-02 20:00:43 首次发布

本文链接：https://blog.csdn.net/m0_49397655/article/details/129307971

版权

python高阶专栏收录该内容

2 篇文章 0 订阅

订阅专栏

并发编程-线程

一个进程是操作系统中运行的一个任务，进程独立拥有CPU、内存等资源
一个线程是一个进程中运行的一个任务，线程之间共享CPU、内存等资源，进程里的每一个任务都是线程。

线程创建

创建线程：使用threading模块中的Thread类来创建线程

target表示线程执行的任务
args表示任务的参数，是一个元组
start()方法指启动线程
join()方法指等待线程结束

from threading import Thread


def task(count:int):
    ...


thread1 = Thread(target=task, args=(10,))  # 创建子线程thread1
thread1.start()  # 启动子线程

启动两个线程执行task任务

from threading import Thread


def task(count: int):
    for i in range(count):
        print(i)


thread1 = Thread(target=task, args=(10,))
thread2 = Thread(target=task, args=(20,))

thread1.start()  # 启动线程1
thread2.start()	 # 启动线程2
print("main thread is end")  # 执行到此处，主线程无任务，自行结束

输出结果：

0
1
2
3
4
5
06main thread is end

7

8
9
1
2
3
4
5
6
7
8
9

等待主线程结束：

from threading import Thread


def task():
    for i in range(10):
        print(i)


thread1 = Thread(target=task)
thread2 = Thread(target=task)

thread1.start()
thread2.start()
thread1.join()  # 等待线程执行结束
thread2.join()  # 等待线程执行结束
print("main thread is end")  # 等到线程1和线程2任务执行完毕，主线程才会结束

输出结果：

0
1
2
3
4
50
6
7
8
9
1

2
3
4
5
6
7
8
9
main thread is end

通过继承创建线程

通过继承Thread类来创建线程

打印混乱的原因：多线程执行时，print函数打印内容后会再打印换行，导致换行来不及打印，出现打印混乱

解决办法：行尾手动加换行，print函数end参数设置为空字符串
print(f"{self.name} - {i}\n", end="")

from threading import Thread


class MyThread(Thread):
    def run(self) -> None:
        # 线程的任务脚本
        pass

import time
from threading import Thread


class MyThread(Thread):

    def __init__(self, name: str, count: int):
        super().__init__()
        # 该方法已弃用，使用name属性替换
        # self.setName()
        self.name = name  # 设置线程名称
        self.count = count

    def run(self) -> None:
        # 线程的任务脚本
        for i in range(self.count):
            print(f"{self.name} - {i}\n", end="")
            # 休眠10毫秒
            time.sleep(0.01)


t1 = MyThread("a", 10)
t2 = MyThread("b", 10)
t1.start()
t2.start()

输出结果：

D:\Python3.11\python.exe F:/Code/python_code/high_python/python_advance/thread/demo01_class.py
a - 0
b - 0
a - 1
b - 1
b - 2
a - 2
a - 3
b - 3
b - 4
a - 4
b - 5
a - 5
b - 6
a - 6
b - 7
a - 7
b - 8
a - 8
b - 9
a - 9

守护线程

主线程结束，守护线程会自动结束，这就叫守护线程

守护线程会在主线程结束时候自动结束
主线程则需要等待到所有的非守护线程结束才能结束
守护线程一般用于非关键性的线程，比如：日志

from threading import Thread


def task():
    for i in range(10):
        print(i)


thread = Thread(target=task, daemon=True)
thread.start()

输出结果：

0
Process finished with exit code 0

等待线程结束

from threading import Thread


def task():
    for i in range(10):
        print(i)


thread = Thread(target=task, daemon=True)
thread.start()
thread.join()

输出结果：

D:\Python3.11\python.exe F:/Code/python_code/high_python/python_advance/thread/demo03_daemon_thread.py
0
1
2
3
4
5
6
7
8
9

Process finished with exit code 0

继承类设置守护线程

class MyThread(Thread):

    def __init__(self, name: str, count: int):
        super().__init__()
        # 该方法已弃用，使用name属性替换
        # self.setName()
        self.name = name  # 设置线程名称
        self.count = count
        self.daemon = True

    def run(self) -> None:
        # 线程的任务脚本
        for i in range(self.count):
            print(f"{self.name} - {i}\n", end="")
            # 休眠10毫秒
            time.sleep(0.01)


t1 = MyThread("a", 10)
t2 = MyThread("b", 10)
t1.start()
t2.start()
t1.join() # 等待t1结束，结束主线程

线程安全队列

queue模块中的Queue类提供了线程安全队列功能

put(item, block=False) # 队列元素满时，不阻塞，会抛出异常，为True时，会让线程阻塞等待，可能会导致线程卡死
put(item, timeout=3) # 队列元素满时，等待timeout时间，如果超出时间则抛出异常
get(block=False) # 从队列取元素，如果队列为空则不阻塞，会抛出异常
get(timeout=10) # 从队列取元素，如果队列为空则则赛等待10秒，超时抛出异常
qsize() # 队列大小
empty() # 判断队列是否为空
full() # 判断队列是否满的

生产者消费者线程实例

from threading import Thread
from queue import Queue


class Producer(Thread):
	"""生产者"""
    def __init__(self, name: str, count: int, queue: Queue) -> None:
        super(Producer, self).__init__()

        # 线程自带属性
        self.name = name

        # 自定义属性
        self.count = count
        self.queue = queue

    def run(self) -> None:
        for n in range(self.count):
            msg = f"{self.name} - {n}"
            self.queue.put(msg, block=True)


class Consumer(Thread):
	"""消费者"""
    def __init__(self, name: str, queue: Queue) -> None:
        super().__init__()

        self.name = name
        self.daemon = True

        self.queue = queue

    def run(self) -> None:
        while True:
            msg = self.queue.get(block=True)
            print(f"{self.name} - {msg}\n", end="")


queue = Queue(maxsize=3)
threads = [
    Producer("p1", 10, queue),
    Producer("p2", 10, queue),
    Producer("p3", 10, queue),

    Consumer("c1", queue),
    Consumer("c2", queue),
]

for t in threads:
    t.start()

线程池

线程的创建和销毁相对比较耗费资源
频繁的创建和销毁线程不利于高性能
线程池是python提供的便于线程管理和提高性能的工具

python提供两个类来管理线程

concurrent.futures.ThreadPoolExecutor
- submit() # 启动/执行一个任务，返回结果是一个Future对象
- map() # 多个任务执行，将不同参数分配到每一个任务中
- shutdown() # 关闭线程池
Future
- result() # 任务执行结果
- exception() # 任务异常信息

方式一：适用于不同任务

from concurrent.futures import ThreadPoolExecutor
import time


def task(name: str):
    print(f"{name} - step 1\n", end="")
    time.sleep(1)
    print(f"{name} - step 2\n", end="")
    
    return f"{name} complete"
    
    
with ThreadPoolExecutor() as executor:
    result_1 = executor.submit(task, "A")
    result_2 = executor.submit(task, "B")
    
    print(result_1.result())  # result()会等待有结果再返回
    print(result_2.result())
    
"""
A - step 1
B - step 1
A - step 2
A complete
B - step 2
B complete
"""

方式二：map()适用于同一任务，不同参数

from concurrent.futures import ThreadPoolExecutor
import time

def task(name: str) -> str:
    print(f"{name} - step 1\n", end="")
    time.sleep(1)
    print(f"{name} - step 2\n", end="")
    
    return f"{name} complete"


with ThreadPoolExecutor() as executor:
    results = executor.map(task, ["C", "D"])
    for r in results:
        print(r)
        
"""
C - step 1
D - step 1
D - step 2
C - step 2
C complete
D complete
"""

多线程案例：下载图片

from concurrent.futures import ThreadPoolExecutor
from urllib.request import urlopen, Request
import os

def download_img(url: str):
    site_url = Request(url, headers={})

    with urlopen(site_url) as web_file:
        img_data = web_file.read()
        
    if not img_data:
        raise Exception(f"Error: can not load the image from {url}")
    
    file_name = os.path.basename(url)
    
    with open(file_name, "wb") as f:
        f.write(img_data)
        
    return "Download image successfully, {}".format(url)


urls = [
    "https://img0.bdstatic.com/static/searchresult/img/baseimg3_4f26a23.png",
    # "..."
]
headers = {
    "User-Agent": "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/110.0.0.0 Safari/537.36",
    "content-type": "Content-Type: image/jpeg"
}
with ThreadPoolExecutor() as ececutor:
    results = ececutor.map(download_img, urls)
    for r in results:
        print(r)

执行结果：

PS F:\Code\python_code\high_python\python_advance\thread> python .\thread_pool_demo.py
Download image successfully, https://img0.bdstatic.com/static/searchresult/img/baseimg3_4f26a23.png