python并发与并行（六） ———— 正确的重构代码，以便用Queue做并发

最新推荐文章于 2024-08-30 12:42:13 发布

bug404_

最新推荐文章于 2024-08-30 12:42:13 发布

阅读量843

点赞数 14

分类专栏： python并发与并行文章标签： python 重构开发语言

本文链接：https://blog.csdn.net/bornfree5511/article/details/141651353

版权

python并发与并行专栏收录该内容

13 篇文章 0 订阅

订阅专栏

在前面“python并发与并行（五.2） ———— 不要在每次fan-out时都新建一批Thread实例”里面，大家看到，每次都手工创建一批线程并平行地执行I/O任务是有很多缺点的。这一条要介绍另一种方案，也就是用内置的queue模块里的Queue类实现多线程管道。

Queue方案的总思路是：在推进游戏时，不像原来那样，每推进一代，就新建一批线程来推进相应的单元格，而是可以提前创建数量固定的一组工作线程，令这组线程平行地处理当前这批I/O任务，并在处理完之后，继续等待下一批任务，这样就不会消耗那么多资源了，程序也不会再因为频繁新建线程而耽误那么多时间。

首先，创建两个ClosableQueue实例表示输入队列（in_queue）与输出队列（out_queue）。每条工作线程都可以从输入队列里面取出有待执行的game_logic任务，并把执行结果放到输出队列之中。


from queue import Queue

class ClosableQueue(Queue):
    # 定义哨兵对象
    # 哨兵（SENTINEL）是一个唯一的对象，用于作为队列关闭的信号。
    # 哨兵对象是不可变的，并且是唯一的，这意味着在程序中不会有其他对象与之相同。
    SENTINEL = object()

    # close 方法将哨兵对象放入队列中。当线程从队列中取出哨兵对象时，
    # 它会识别出队列已被关闭，并可以采取适当的行动，比如退出循环或终止线程。
    def close(self):
        self.put(self.SENTINEL)

    def __iter__(self):
        while True:
            item = self.get()
            try:
                if item is self.SENTINEL:
                    return  # Cause the thread to exit
                yield item
            finally:
                # task_done 方法用于通知队列某个任务已经处理完成，
                # 这对于队列的消费者线程是必要的，以确保队列知道何时所有项目都已被处理。
                self.task_done()

in_queue = ClosableQueue()
out_queue = ClosableQueue()

使用 ClosableQueue 类可以方便地在多线程环境中控制数据流，当主线程决定不再处理数据时，可以通过调用 close 方法来通知工作线程停止处理并安全退出。这种模式在生产者-消费者问题中特别有用。

然后我们启动多线程从in_queue里面获取任务，并调用game_logic来处理各自的任务，然后把结果放到out_queue里面。这些线程会并发地运行，从而平行地处理这些I/O任务，以降低游戏进入下一代时的延迟。

from threading import Thread

class StoppableWorker(Thread):
    def __init__(self, func, in_queue, out_queue, **kwargs):
        super().__init__(**kwargs)
        self.func = func
        self.in_queue = in_queue
        self.out_queue = out_queue

    def run(self):
        for item in self.in_queue:
            result = self.func(item)
            self.out_queue.put(result)

# def game_logic(state, neighbors):
#     # Do some blocking input/output in here:
#     data = my_socket.recv(100)
ALIVE = '*'
EMPTY = '-'
def game_logic(state, neighbors):
    if state == ALIVE:
        if neighbors < 2:
            return EMPTY     # Die: Too few
        elif neighbors > 3:
            return EMPTY     # Die: Too many
    else:
        if neighbors == 3:
            return ALIVE     # Regenerate
    return state

def game_logic_thread(item):
    y, x, state, neighbors = item
    try:
        next_state = game_logic(state, neighbors)
    except Exception as e:
        next_state = e
    return (y, x, next_state)

# Start the threads upfront
threads = []
for _ in range(5):
    thread = StoppableWorker(
        game_logic_thread, in_queue, out_queue)
    thread.start()
    threads.append(thread)

simulate函数需要重构。这次，它必须与刚才的两个队列交互，把需要迁移的每一个单元格及其当前状态都封装成一项任务放到输入队列之中，并等待那些线程把输入队列中的所有任务都处理完，接下来，它要根据输出队列所收集到的结果更新每个单元格的状态，从而将游戏推进到下一代。把迁移单元格状态的任务放到输入队列，派发给工作线程去执行，这里用的是fan-out（扇出）模式；等待这些任务全都处理完毕，然后才开始使用输出队列所收集到的结果，这里用的是fan-in（扇入）模式。


class SimulationError(Exception):
    pass

class Grid:
    def __init__(self, height, width):
        self.height = height
        self.width = width
        self.rows = []
        for _ in range(self.height):
            self.rows.append([EMPTY] * self.width)

    def get(self, y, x):
        return self.rows[y % self.height][x % self.width]

    def set(self, y, x, state):
        self.rows[y % self.height][x % self.width] = state

    def __str__(self):
        output = ''
        for row in self.rows:
            for cell in row:
                output += cell
            output += '\n'
        return output

def count_neighbors(y, x, get):
    n_ = get(y - 1, x + 0)  # North
    ne = get(y - 1, x + 1)  # Northeast
    e_ = get(y + 0, x + 1)  # East
    se = get(y + 1, x + 1)  # Southeast
    s_ = get(y + 1, x + 0)  # South
    sw = get(y + 1, x - 1)  # Southwest
    w_ = get(y + 0, x - 1)  # West
    nw = get(y - 1, x - 1)  # Northwest
    neighbor_states = [n_, ne, e_, se, s_, sw, w_, nw]
    count = 0
    for state in neighbor_states:
        if state == ALIVE:
            count += 1
    return count

def simulate_pipeline(grid, in_queue, out_queue):
    for y in range(grid.height):
        for x in range(grid.width):
            state = grid.get(y, x)
            neighbors = count_neighbors(y, x, grid.get)
            in_queue.put((y, x, state, neighbors))  # Fan out

    in_queue.join()
    out_queue.close()

    next_grid = Grid(grid.height, grid.width)
    for item in out_queue:                          # Fan in
        y, x, next_state = item
        if isinstance(next_state, Exception):
            raise SimulationError(y, x) from next_state
        next_grid.set(y, x, next_state)

    return next_grid

新的simulate_pipeline函数要用到网格（Grid）类的get方法与set方法，但它这次是在自身所处的这条线程里面调用这两个方法的，所以不用像上一条那样，专门编写一种支持多线程的网格，并通过Lock机制给这两个方法加锁以防网格之中的数据遭到破坏。我们这次可以直接使用普通版的Grid类。

完整的代码如下：

import logging
from queue import Queue

class ClosableQueue(Queue):
    SENTINEL = object()

    def close(self):
        self.put(self.SENTINEL)

    def __iter__(self):
        while True:
            item = self.get()
            try:
                if item is self.SENTINEL:
                    return  # Cause the thread to exit
                yield item
            finally:
                self.task_done()

in_queue = ClosableQueue()
out_queue = ClosableQueue()


from threading import Thread

class StoppableWorker(Thread):
    def __init__(self, func, in_queue, out_queue, **kwargs):
        super().__init__(**kwargs)
        self.func = func
        self.in_queue = in_queue
        self.out_queue = out_queue

    def run(self):
        for item in self.in_queue:
            result = self.func(item)
            self.out_queue.put(result)

# def game_logic(state, neighbors):
#     # Do some blocking input/output in here:
#     data = my_socket.recv(100)

ALIVE = '*'
EMPTY = '-'
def game_logic(state, neighbors):
    if state == ALIVE:
        if neighbors < 2:
            return EMPTY     # Die: Too few
        elif neighbors > 3:
            return EMPTY     # Die: Too many
    else:
        if neighbors == 3:
            return ALIVE     # Regenerate
    return state

def game_logic_thread(item):
    y, x, state, neighbors = item
    try:
        next_state = game_logic(state, neighbors)
    except Exception as e:
        next_state = e
    return (y, x, next_state)

# Start the threads upfront

threads = []
for _ in range(5):
    thread = StoppableWorker(
        game_logic_thread, in_queue, out_queue)
    thread.start()
    threads.append(thread)


class SimulationError(Exception):
    pass

class Grid:
    def __init__(self, height, width):
        self.height = height
        self.width = width
        self.rows = []
        for _ in range(self.height):
            self.rows.append([EMPTY] * self.width)

    def get(self, y, x):
        return self.rows[y % self.height][x % self.width]

    def set(self, y, x, state):
        self.rows[y % self.height][x % self.width] = state

    def __str__(self):
        output = ''
        for row in self.rows:
            for cell in row:
                output += cell
            output += '\n'
        return output

def count_neighbors(y, x, get):
    n_ = get(y - 1, x + 0)  # North
    ne = get(y - 1, x + 1)  # Northeast
    e_ = get(y + 0, x + 1)  # East
    se = get(y + 1, x + 1)  # Southeast
    s_ = get(y + 1, x + 0)  # South
    sw = get(y + 1, x - 1)  # Southwest
    w_ = get(y + 0, x - 1)  # West
    nw = get(y - 1, x - 1)  # Northwest
    neighbor_states = [n_, ne, e_, se, s_, sw, w_, nw]
    count = 0
    for state in neighbor_states:
        if state == ALIVE:
            count += 1
    return count

def simulate_pipeline(grid, in_queue, out_queue):
    for y in range(grid.height):
        for x in range(grid.width):
            state = grid.get(y, x)
            neighbors = count_neighbors(y, x, grid.get)
            in_queue.put((y, x, state, neighbors))  # Fan out

    in_queue.join()
    out_queue.close()

    next_grid = Grid(grid.height, grid.width)
    for item in out_queue:                          # Fan in
        y, x, next_state = item
        if isinstance(next_state, Exception):
            raise SimulationError(y, x) from next_state
        next_grid.set(y, x, next_state)

    return next_grid


# try:
#     def game_logic(state, neighbors):
#         raise OSError('Problem with I/O in game_logic')
#
#
#     simulate_pipeline(Grid(1, 1), in_queue, out_queue)
# except:
#     logging.exception('Expected')
# else:
#     assert False


class ColumnPrinter:
    def __init__(self):
        self.columns = []

    def append(self, data):
        self.columns.append(data)

    def __str__(self):
        row_count = 1
        for data in self.columns:
            row_count = max(
                row_count, len(data.splitlines()) + 1)

        rows = [''] * row_count
        for j in range(row_count):
            for i, data in enumerate(self.columns):
                line = data.splitlines()[max(0, j - 1)]
                if j == 0:
                    padding = ' ' * (len(line) // 2)
                    rows[j] += padding + str(i) + padding
                else:
                    rows[j] += line

                if (i + 1) < len(self.columns):
                    rows[j] += ' | '

        return '\n'.join(rows)

grid = Grid(5, 9)
grid.set(0, 3, ALIVE)
grid.set(1, 4, ALIVE)
grid.set(2, 2, ALIVE)
grid.set(2, 3, ALIVE)
grid.set(2, 4, ALIVE)

columns = ColumnPrinter()
for i in range(5):
    columns.append(str(grid))
    grid = simulate_pipeline(grid, in_queue, out_queue)

print(columns)

for thread in threads:
    in_queue.close()
for thread in threads:
    thread.join()

现在的代码要比那种每次都手工新建一批线程的方案更容易调试。如果game_logic函数执行I/O时出错，那么错误会被捕获并传入输出队列里面，并在主线程之中重新抛出。

这样写的效果和原来一样。与那种每次操作都启动一条线程的做法相比，这种方案使用的内存少，启动线程时的开销也小，而且调试起来较为容易。尽管如此，但它还是有几个问题：

负责把网格推进到下一代的simulate_pipeline函数，要比上一条里面的对应函数simulate_threaded难懂

为了让代码更容易理解，必须用ClosableQueue与StoppableWorker这样的类表示特制的队列与工作线程，这会让程序变得复杂。

与并行度有关的那个参数（即运行game_logic_thread函数的线程数量），必须依工作负载提前定好，而无法由系统根据工作量自动调整。

我们同样像前几节那样，把Grid的大小改为（50，90），然后迭代100次，看一下性能。

主要是修改如下的地方：

grid = Grid(50, 90)
grid.set(0, 3, ALIVE)
grid.set(1, 4, ALIVE)
grid.set(2, 2, ALIVE)
grid.set(2, 3, ALIVE)
grid.set(2, 4, ALIVE)

columns = ColumnPrinter()
import time
st=time.time()
for i in range(100):
    columns.append(str(grid))
    grid = simulate_pipeline(grid, in_queue, out_queue)

print(f'::LOG::simulation time: {time.time() - st}')

全部的代码如下：

import logging
from queue import Queue

class ClosableQueue(Queue):
    SENTINEL = object()

    def close(self):
        self.put(self.SENTINEL)

    def __iter__(self):
        while True:
            item = self.get()
            try:
                if item is self.SENTINEL:
                    return  # Cause the thread to exit
                yield item
            finally:
                self.task_done()

in_queue = ClosableQueue()
out_queue = ClosableQueue()


from threading import Thread

class StoppableWorker(Thread):
    def __init__(self, func, in_queue, out_queue, **kwargs):
        super().__init__(**kwargs)
        self.func = func
        self.in_queue = in_queue
        self.out_queue = out_queue

    def run(self):
        for item in self.in_queue:
            result = self.func(item)
            self.out_queue.put(result)

# def game_logic(state, neighbors):
#     # Do some blocking input/output in here:
#     data = my_socket.recv(100)

ALIVE = '*'
EMPTY = '-'
def game_logic(state, neighbors):
    if state == ALIVE:
        if neighbors < 2:
            return EMPTY     # Die: Too few
        elif neighbors > 3:
            return EMPTY     # Die: Too many
    else:
        if neighbors == 3:
            return ALIVE     # Regenerate
    return state

def game_logic_thread(item):
    y, x, state, neighbors = item
    try:
        next_state = game_logic(state, neighbors)
    except Exception as e:
        next_state = e
    return (y, x, next_state)

# Start the threads upfront

threads = []
for _ in range(5):
    thread = StoppableWorker(
        game_logic_thread, in_queue, out_queue)
    thread.start()
    threads.append(thread)


class SimulationError(Exception):
    pass

class Grid:
    def __init__(self, height, width):
        self.height = height
        self.width = width
        self.rows = []
        for _ in range(self.height):
            self.rows.append([EMPTY] * self.width)

    def get(self, y, x):
        return self.rows[y % self.height][x % self.width]

    def set(self, y, x, state):
        self.rows[y % self.height][x % self.width] = state

    def __str__(self):
        output = ''
        for row in self.rows:
            for cell in row:
                output += cell
            output += '\n'
        return output

def count_neighbors(y, x, get):
    n_ = get(y - 1, x + 0)  # North
    ne = get(y - 1, x + 1)  # Northeast
    e_ = get(y + 0, x + 1)  # East
    se = get(y + 1, x + 1)  # Southeast
    s_ = get(y + 1, x + 0)  # South
    sw = get(y + 1, x - 1)  # Southwest
    w_ = get(y + 0, x - 1)  # West
    nw = get(y - 1, x - 1)  # Northwest
    neighbor_states = [n_, ne, e_, se, s_, sw, w_, nw]
    count = 0
    for state in neighbor_states:
        if state == ALIVE:
            count += 1
    return count

def simulate_pipeline(grid, in_queue, out_queue):
    for y in range(grid.height):
        for x in range(grid.width):
            state = grid.get(y, x)
            neighbors = count_neighbors(y, x, grid.get)
            in_queue.put((y, x, state, neighbors))  # Fan out

    in_queue.join()
    out_queue.close()

    next_grid = Grid(grid.height, grid.width)
    for item in out_queue:                          # Fan in
        y, x, next_state = item
        if isinstance(next_state, Exception):
            raise SimulationError(y, x) from next_state
        next_grid.set(y, x, next_state)

    return next_grid


# try:
#     def game_logic(state, neighbors):
#         raise OSError('Problem with I/O in game_logic')
#
#
#     simulate_pipeline(Grid(1, 1), in_queue, out_queue)
# except:
#     logging.exception('Expected')
# else:
#     assert False


class ColumnPrinter:
    def __init__(self):
        self.columns = []

    def append(self, data):
        self.columns.append(data)

    def __str__(self):
        row_count = 1
        for data in self.columns:
            row_count = max(
                row_count, len(data.splitlines()) + 1)

        rows = [''] * row_count
        for j in range(row_count):
            for i, data in enumerate(self.columns):
                line = data.splitlines()[max(0, j - 1)]
                if j == 0:
                    padding = ' ' * (len(line) // 2)
                    rows[j] += padding + str(i) + padding
                else:
                    rows[j] += line

                if (i + 1) < len(self.columns):
                    rows[j] += ' | '

        return '\n'.join(rows)

grid = Grid(50, 90)
grid.set(0, 3, ALIVE)
grid.set(1, 4, ALIVE)
grid.set(2, 2, ALIVE)
grid.set(2, 3, ALIVE)
grid.set(2, 4, ALIVE)

columns = ColumnPrinter()
import time
st=time.time()
for i in range(100):
    columns.append(str(grid))
    grid = simulate_pipeline(grid, in_queue, out_queue)

print(f'::LOG::simulation time: {time.time() - st}')

print(columns)

for thread in threads:
    in_queue.close()
for thread in threads:
    thread.join()

Output:

::LOG::simulation time: 2.5422348976135254

我们从前一节得知，前一节实现的多线程 simulation time: 14.29249095916748 单线程 simulate time: 0.6782510280609131。

我们本节的实现虽然比前一节的多线程版本有较大的性能提升，但是仍然还不如单线程的版本。

threads = []
for _ in range(10):
    thread = StoppableWorker(
        game_logic_thread, in_queue, out_queue)
    thread.start()
    threads.append(thread)

如上面，我们把总的线程增加到10个，::LOG::simulation time: 3.507694721221924

我们把线程数设为1个，::LOG::simulation time: 2.471439838409424

我们把线程数设为20个，::LOG::simulation time: 3.195152759552002

如果我们把Grid设置为（500，900），迭代次数为100次，线程数设为5，我们看一下耗时。

grid = Grid(500, 900)
grid.set(0, 3, ALIVE)
grid.set(1, 4, ALIVE)
grid.set(2, 2, ALIVE)
grid.set(2, 3, ALIVE)
grid.set(2, 4, ALIVE)

columns = ColumnPrinter()
import time
from tqdm import tqdm
st=time.time()

for i in tqdm(range(100)):
    columns.append(str(grid))
    grid = simulate_pipeline(grid, in_queue, out_queue)

print(f'::LOG::simulation time: {time.time() - st}')

::LOG::simulation time: 170.44810271263123

我们把前面的单线程的程序，用同样的规格跑一下。

ALIVE = '*'
EMPTY = '-'

class Grid:
    def __init__(self, height, width):
        self.height = height
        self.width = width
        self.rows = []
        for _ in range(self.height):
            self.rows.append([EMPTY] * self.width)

    def get(self, y, x):
        return self.rows[y % self.height][x % self.width]

    def set(self, y, x, state):
        self.rows[y % self.height][x % self.width] = state

    def __str__(self):
        output = ''
        for row in self.rows:
            for cell in row:
                output += cell
            output += '\n'
        return output


grid = Grid(500, 900)
grid.set(0, 3, ALIVE)
grid.set(1, 4, ALIVE)
grid.set(2, 2, ALIVE)
grid.set(2, 3, ALIVE)
grid.set(2, 4, ALIVE)
print(grid)


def count_neighbors(y, x, get):
    n_ = get(y - 1, x + 0)  # North
    ne = get(y - 1, x + 1)  # Northeast
    e_ = get(y + 0, x + 1)  # East
    se = get(y + 1, x + 1)  # Southeast
    s_ = get(y + 1, x + 0)  # South
    sw = get(y + 1, x - 1)  # Southwest
    w_ = get(y + 0, x - 1)  # West
    nw = get(y - 1, x - 1)  # Northwest
    neighbor_states = [n_, ne, e_, se, s_, sw, w_, nw]
    count = 0
    for state in neighbor_states:
        if state == ALIVE:
            count += 1
    return count

alive = {(9, 5), (9, 6)}
seen = set()

def fake_get(y, x):
    position = (y, x)
    seen.add(position)
    return ALIVE if position in alive else EMPTY

count = count_neighbors(10, 5, fake_get)
assert count == 2

expected_seen = {
    (9, 5),  (9, 6),  (10, 6), (11, 6),
    (11, 5), (11, 4), (10, 4), (9, 4)
}
assert seen == expected_seen


def game_logic(state, neighbors):
    if state == ALIVE:
        if neighbors < 2:
            return EMPTY     # Die: Too few
        elif neighbors > 3:
            return EMPTY     # Die: Too many
    else:
        if neighbors == 3:
            return ALIVE     # Regenerate
    return state

assert game_logic(ALIVE, 0) == EMPTY
assert game_logic(ALIVE, 1) == EMPTY
assert game_logic(ALIVE, 2) == ALIVE
assert game_logic(ALIVE, 3) == ALIVE
assert game_logic(ALIVE, 4) == EMPTY
assert game_logic(EMPTY, 0) == EMPTY
assert game_logic(EMPTY, 1) == EMPTY
assert game_logic(EMPTY, 2) == EMPTY
assert game_logic(EMPTY, 3) == ALIVE
assert game_logic(EMPTY, 4) == EMPTY


def step_cell(y, x, get, set):
    state = get(y, x)
    neighbors = count_neighbors(y, x, get)
    next_state = game_logic(state, neighbors)
    set(y, x, next_state)

alive = {(10, 5), (9, 5), (9, 6)}
new_state = None

def fake_get(y, x):
    return ALIVE if (y, x) in alive else EMPTY

def fake_set(y, x, state):
    global new_state
    new_state = state

# Stay alive
step_cell(10, 5, fake_get, fake_set)
assert new_state == ALIVE

# Stay dead
alive.remove((10, 5))
step_cell(10, 5, fake_get, fake_set)
assert new_state == EMPTY

# Regenerate
alive.add((10, 6))
step_cell(10, 5, fake_get, fake_set)
assert new_state == ALIVE


def simulate(grid):
    next_grid = Grid(grid.height, grid.width)
    for y in range(grid.height):
        for x in range(grid.width):
            step_cell(y, x, grid.get, next_grid.set)
    return next_grid


class ColumnPrinter:
    def __init__(self):
        self.columns = []

    def append(self, data):
        self.columns.append(data)

    def __str__(self):
        row_count = 1
        for data in self.columns:
            row_count = max(
                row_count, len(data.splitlines()) + 1)

        rows = [''] * row_count
        for j in range(row_count):
            for i, data in enumerate(self.columns):
                line = data.splitlines()[max(0, j - 1)]
                if j == 0:
                    padding = ' ' * (len(line) // 2)
                    rows[j] += padding + str(i) + padding
                else:
                    rows[j] += line

                if (i + 1) < len(self.columns):
                    rows[j] += ' | '

        return '\n'.join(rows)

columns = ColumnPrinter()
import time
st=time.time()
from tqdm import tqdm
for i in tqdm(range(100)):
    columns.append(str(grid))
    grid = simulate(grid)
print(f'simulate time: {time.time() - st}')
# print(columns)

simulate time: 55.45792198181152

我们继续把Grid大小设置为(900,1200)，看一下对比。

本节实现的多线程：simulation time: 419.4681279659271
单线程：simulate time: 137.2363736629486

多线程的性能仍然不如单线程。我们还使用上一节的cProfile对多线程的代码进行一下profiling，看一下性能瓶颈在什么地方。

对代码做一下修改，将run simulation的部分封装为函数。


import cProfile
import pstats

def run_simulation():
    grid = Grid(50, 90)
    grid.set(0, 3, ALIVE)
    grid.set(1, 4, ALIVE)
    grid.set(2, 2, ALIVE)
    grid.set(2, 3, ALIVE)
    grid.set(2, 4, ALIVE)

    columns = ColumnPrinter()
    import time
    from tqdm import tqdm
    st=time.time()

    for i in tqdm(range(100)):
        columns.append(str(grid))
        grid = simulate_pipeline(grid, in_queue, out_queue)

    print(f'::LOG::simulation time: {time.time() - st}')

cProfile.run('run_simulation()','threaded_prof')
with open('threaded_profile.txt', 'w') as f:
    p = pstats.Stats('threaded_prof', stream=f)
    p.sort_stats('cumulative').print_stats()

print("Profile saved to 'threaded_profile.txt'")

完整代码为：

import logging
from queue import Queue

class ClosableQueue(Queue):
    SENTINEL = object()

    def close(self):
        self.put(self.SENTINEL)

    def __iter__(self):
        while True:
            item = self.get()
            try:
                if item is self.SENTINEL:
                    return  # Cause the thread to exit
                yield item
            finally:
                self.task_done()

in_queue = ClosableQueue()
out_queue = ClosableQueue()


from threading import Thread

class StoppableWorker(Thread):
    def __init__(self, func, in_queue, out_queue, **kwargs):
        super().__init__(**kwargs)
        self.func = func
        self.in_queue = in_queue
        self.out_queue = out_queue

    def run(self):
        for item in self.in_queue:
            result = self.func(item)
            self.out_queue.put(result)

# def game_logic(state, neighbors):
#     # Do some blocking input/output in here:
#     data = my_socket.recv(100)

ALIVE = '*'
EMPTY = '-'
def game_logic(state, neighbors):
    if state == ALIVE:
        if neighbors < 2:
            return EMPTY     # Die: Too few
        elif neighbors > 3:
            return EMPTY     # Die: Too many
    else:
        if neighbors == 3:
            return ALIVE     # Regenerate
    return state

def game_logic_thread(item):
    y, x, state, neighbors = item
    try:
        next_state = game_logic(state, neighbors)
    except Exception as e:
        next_state = e
    return (y, x, next_state)

# Start the threads upfront

threads = []
for _ in range(10):
    thread = StoppableWorker(
        game_logic_thread, in_queue, out_queue)
    thread.start()
    threads.append(thread)


class SimulationError(Exception):
    pass

class Grid:
    def __init__(self, height, width):
        self.height = height
        self.width = width
        self.rows = []
        for _ in range(self.height):
            self.rows.append([EMPTY] * self.width)

    def get(self, y, x):
        return self.rows[y % self.height][x % self.width]

    def set(self, y, x, state):
        self.rows[y % self.height][x % self.width] = state

    def __str__(self):
        output = ''
        for row in self.rows:
            for cell in row:
                output += cell
            output += '\n'
        return output

def count_neighbors(y, x, get):
    n_ = get(y - 1, x + 0)  # North
    ne = get(y - 1, x + 1)  # Northeast
    e_ = get(y + 0, x + 1)  # East
    se = get(y + 1, x + 1)  # Southeast
    s_ = get(y + 1, x + 0)  # South
    sw = get(y + 1, x - 1)  # Southwest
    w_ = get(y + 0, x - 1)  # West
    nw = get(y - 1, x - 1)  # Northwest
    neighbor_states = [n_, ne, e_, se, s_, sw, w_, nw]
    count = 0
    for state in neighbor_states:
        if state == ALIVE:
            count += 1
    return count

def simulate_pipeline(grid, in_queue, out_queue):
    for y in range(grid.height):
        for x in range(grid.width):
            state = grid.get(y, x)
            neighbors = count_neighbors(y, x, grid.get)
            in_queue.put((y, x, state, neighbors))  # Fan out

    in_queue.join()
    out_queue.close()

    next_grid = Grid(grid.height, grid.width)
    for item in out_queue:                          # Fan in
        y, x, next_state = item
        if isinstance(next_state, Exception):
            raise SimulationError(y, x) from next_state
        next_grid.set(y, x, next_state)

    return next_grid


# try:
#     def game_logic(state, neighbors):
#         raise OSError('Problem with I/O in game_logic')
#
#
#     simulate_pipeline(Grid(1, 1), in_queue, out_queue)
# except:
#     logging.exception('Expected')
# else:
#     assert False


class ColumnPrinter:
    def __init__(self):
        self.columns = []

    def append(self, data):
        self.columns.append(data)

    def __str__(self):
        row_count = 1
        for data in self.columns:
            row_count = max(
                row_count, len(data.splitlines()) + 1)

        rows = [''] * row_count
        for j in range(row_count):
            for i, data in enumerate(self.columns):
                line = data.splitlines()[max(0, j - 1)]
                if j == 0:
                    padding = ' ' * (len(line) // 2)
                    rows[j] += padding + str(i) + padding
                else:
                    rows[j] += line

                if (i + 1) < len(self.columns):
                    rows[j] += ' | '

        return '\n'.join(rows)


import cProfile
import pstats

def run_simulation():
    grid = Grid(50, 90)
    grid.set(0, 3, ALIVE)
    grid.set(1, 4, ALIVE)
    grid.set(2, 2, ALIVE)
    grid.set(2, 3, ALIVE)
    grid.set(2, 4, ALIVE)

    columns = ColumnPrinter()
    import time
    from tqdm import tqdm
    st=time.time()

    for i in tqdm(range(100)):
        columns.append(str(grid))
        grid = simulate_pipeline(grid, in_queue, out_queue)

    print(f'::LOG::simulation time: {time.time() - st}')

cProfile.run('run_simulation()','threaded_prof')
with open('threaded_profile.txt', 'w') as f:
    p = pstats.Stats('threaded_prof', stream=f)
    p.sort_stats('cumulative').print_stats()

print("Profile saved to 'threaded_profile.txt'")

for thread in threads:
    in_queue.close()
for thread in threads:
    thread.join()

结果为：

Fri Aug  9 23:09:25 2024    threaded_prof

         18065587 function calls (18065085 primitive calls) in 3.713 seconds

   Ordered by: cumulative time

   ncalls  tottime  percall  cumtime  percall filename:lineno(function)
     39/1    0.000    0.000    3.713    3.713 {built-in method builtins.exec}
        1    0.001    0.001    3.713    3.713 /Users/wenyan/projects2024/efficient_python/test_queue.py:178(run_simulation)
      100    0.226    0.002    3.665    0.037 /Users/wenyan/projects2024/efficient_python/test_queue.py:114(simulate_pipeline)
   450100    0.199    0.000    1.428    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/queue.py:122(put)
  1350401    0.122    0.000    1.111    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/threading.py:256(__enter__)
  1350401    0.989    0.000    0.989    0.000 {method '__enter__' of '_thread.lock' objects}
   450100    0.094    0.000    0.853    0.000 /Users/wenyan/projects2024/efficient_python/test_queue.py:10(__iter__)
   450000    0.380    0.000    0.774    0.000 /Users/wenyan/projects2024/efficient_python/test_queue.py:98(count_neighbors)
   450100    0.216    0.000    0.540    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/queue.py:154(get)
  4050000    0.450    0.000    0.450    0.000 /Users/wenyan/projects2024/efficient_python/test_queue.py:84(get)
   900704    0.313    0.000    0.313    0.000 {method 'acquire' of '_thread.lock' objects}
      100    0.000    0.000    0.269    0.003 /opt/anaconda3/envs/calc_photo/lib/python3.9/queue.py:79(join)
      101    0.000    0.000    0.269    0.003 /opt/anaconda3/envs/calc_photo/lib/python3.9/threading.py:280(wait)
   900300    0.092    0.000    0.222    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/threading.py:351(notify)
   450100    0.110    0.000    0.220    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/queue.py:57(task_done)
  1350401    0.123    0.000    0.173    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/threading.py:259(__exit__)
   900401    0.083    0.000    0.128    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/threading.py:271(_is_owned)
   450100    0.044    0.000    0.056    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/queue.py:217(_get)
   450100    0.042    0.000    0.053    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/queue.py:213(_put)
  1350550    0.050    0.000    0.050    0.000 {method '__exit__' of '_thread.lock' objects}
   450100    0.036    0.000    0.048    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/queue.py:209(_qsize)
   450005    0.043    0.000    0.043    0.000 /Users/wenyan/projects2024/efficient_python/test_queue.py:87(set)
     64/6    0.000    0.000    0.027    0.005 <frozen importlib._bootstrap>:1002(_find_and_load)
     64/6    0.000    0.000    0.027    0.005 <frozen importlib._bootstrap>:967(_find_and_load_unlocked)
     60/6    0.000    0.000    0.027    0.004 <frozen importlib._bootstrap>:659(_load_unlocked)
     91/8    0.000    0.000    0.026    0.003 <frozen importlib._bootstrap>:220(_call_with_frames_removed)
     38/4    0.000    0.000    0.026    0.006 <frozen importlib._bootstrap_external>:844(exec_module)
        1    0.000    0.000    0.020    0.020 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/std.py:663(__new__)
        1    0.000    0.000    0.020    0.020 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/std.py:760(get_lock)
        1    0.000    0.000    0.020    0.020 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/std.py:90(__init__)
        1    0.000    0.000    0.020    0.020 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/std.py:116(create_mp_lock)
      100    0.014    0.000    0.014    0.000 /Users/wenyan/projects2024/efficient_python/test_queue.py:90(__str__)
   452146    0.014    0.000    0.014    0.000 {built-in method builtins.isinstance}
451152/451107    0.012    0.000    0.012    0.000 {built-in method builtins.len}
        1    0.000    0.000    0.012    0.012 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/context.py:70(RLock)
   450100    0.012    0.000    0.012    0.000 {method 'popleft' of 'collections.deque' objects}
     22/9    0.000    0.000    0.011    0.001 <frozen importlib._bootstrap>:1033(_handle_fromlist)
      9/3    0.000    0.000    0.011    0.004 {built-in method builtins.__import__}
   450201    0.011    0.000    0.011    0.000 {method 'append' of 'collections.deque' objects}
       38    0.000    0.000    0.010    0.000 <frozen importlib._bootstrap_external>:916(get_code)
       60    0.000    0.000    0.009    0.000 <frozen importlib._bootstrap>:558(module_from_spec)
       20    0.000    0.000    0.008    0.000 <frozen importlib._bootstrap_external>:1171(create_module)
       20    0.008    0.000    0.008    0.000 {built-in method _imp.create_dynamic}
       38    0.000    0.000    0.008    0.000 <frozen importlib._bootstrap_external>:1036(get_data)
        1    0.000    0.000    0.007    0.007 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/__init__.py:15(<module>)
        1    0.000    0.000    0.007    0.007 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/synchronize.py:10(<module>)
        1    0.000    0.000    0.007    0.007 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/context.py:1(<module>)
        1    0.000    0.000    0.007    0.007 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/__init__.py:1(<module>)
        1    0.000    0.000    0.006    0.006 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/reduction.py:10(<module>)
        1    0.000    0.000    0.005    0.005 /opt/anaconda3/envs/calc_photo/lib/python3.9/tempfile.py:1(<module>)
        1    0.000    0.000    0.005    0.005 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/cli.py:1(<module>)
       38    0.005    0.000    0.005    0.000 {method 'read' of '_io.BufferedReader' objects}
        1    0.000    0.000    0.004    0.004 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/synchronize.py:186(__init__)
        1    0.000    0.000    0.004    0.004 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/synchronize.py:50(__init__)
        1    0.000    0.000    0.004    0.004 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/std.py:1(<module>)
      101    0.000    0.000    0.004    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/std.py:1160(__iter__)
       62    0.000    0.000    0.004    0.000 <frozen importlib._bootstrap>:901(_find_spec)
       31    0.000    0.000    0.004    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/std.py:1325(refresh)
       30    0.000    0.000    0.004    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/std.py:1198(update)
        1    0.000    0.000    0.003    0.003 /opt/anaconda3/envs/calc_photo/lib/python3.9/shutil.py:1(<module>)
       32    0.000    0.000    0.003    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/std.py:1464(display)
       60    0.000    0.000    0.003    0.000 <frozen importlib._bootstrap_external>:1415(find_spec)
       60    0.000    0.000    0.003    0.000 <frozen importlib._bootstrap_external>:1383(_get_spec)
        1    0.000    0.000    0.003    0.003 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/resource_tracker.py:18(<module>)
      129    0.000    0.000    0.003    0.000 <frozen importlib._bootstrap_external>:1514(find_spec)
       38    0.002    0.000    0.002    0.000 {built-in method io.open_code}
        1    0.000    0.000    0.002    0.002 /opt/anaconda3/envs/calc_photo/lib/python3.9/socket.py:4(<module>)
       32    0.000    0.000    0.002    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/std.py:1150(__str__)
        1    0.000    0.000    0.002    0.002 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/spawn.py:11(<module>)
        1    0.000    0.000    0.002    0.002 /opt/anaconda3/envs/calc_photo/lib/python3.9/pickle.py:1(<module>)
       32    0.000    0.000    0.002    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/std.py:464(format_meter)
       38    0.000    0.000    0.002    0.000 <frozen importlib._bootstrap_external>:645(_compile_bytecode)
       38    0.002    0.000    0.002    0.000 {built-in method marshal.loads}
        1    0.000    0.000    0.002    0.002 /opt/anaconda3/envs/calc_photo/lib/python3.9/runpy.py:1(<module>)
     1959    0.002    0.000    0.002    0.000 {method 'release' of '_thread.lock' objects}
        1    0.000    0.000    0.002    0.002 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/std.py:952(__init__)
      235    0.000    0.000    0.002    0.000 <frozen importlib._bootstrap_external>:135(_path_stat)
      235    0.002    0.000    0.002    0.000 {built-in method posix.stat}
       96    0.000    0.000    0.002    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/utils.py:378(disp_len)
       96    0.000    0.000    0.002    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/utils.py:374(_text_width)
       96    0.000    0.000    0.001    0.000 {built-in method builtins.sum}
        1    0.000    0.000    0.001    0.001 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/utils.py:333(_screen_shape_linux)
       64    0.000    0.000    0.001    0.000 <frozen importlib._bootstrap_external>:154(_path_isfile)
       66    0.000    0.000    0.001    0.000 <frozen importlib._bootstrap_external>:145(_path_is_mode_type)
        1    0.000    0.000    0.001    0.001 /opt/anaconda3/envs/calc_photo/lib/python3.9/datetime.py:1(<module>)
        1    0.000    0.000    0.001    0.001 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/utils.py:1(<module>)
      123    0.001    0.000    0.001    0.000 {built-in method builtins.__build_class__}
        1    0.000    0.000    0.001    0.001 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/util.py:10(<module>)
      101    0.001    0.000    0.001    0.000 /Users/wenyan/projects2024/efficient_python/test_queue.py:77(__init__)
       32    0.000    0.000    0.001    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/std.py:457(print_status)
    11925    0.001    0.000    0.001    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/utils.py:375(<genexpr>)
        7    0.000    0.000    0.001    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/enum.py:528(_convert_)
        1    0.000    0.000    0.001    0.001 /opt/anaconda3/envs/calc_photo/lib/python3.9/random.py:1(<module>)
        1    0.000    0.000    0.001    0.001 /opt/anaconda3/envs/calc_photo/lib/python3.9/importlib/util.py:1(<module>)
        1    0.000    0.000    0.001    0.001 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/process.py:10(<module>)
       47    0.000    0.000    0.001    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/enum.py:358(__call__)
        7    0.000    0.000    0.001    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/enum.py:475(_create_)
      111    0.000    0.000    0.001    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/re.py:289(_compile)
       32    0.000    0.000    0.001    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/utils.py:386(disp_trim)
        1    0.000    0.000    0.001    0.001 /opt/anaconda3/envs/calc_photo/lib/python3.9/lzma.py:1(<module>)
        5    0.000    0.000    0.001    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/sre_compile.py:783(compile)
        4    0.000    0.000    0.001    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/re.py:250(compile)
        1    0.000    0.000    0.001    0.001 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/resource_tracker.py:145(register)
        1    0.000    0.000    0.001    0.001 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/resource_tracker.py:153(_send)
        1    0.000    0.000    0.001    0.001 /opt/anaconda3/envs/calc_photo/lib/python3.9/selectors.py:1(<module>)
        1    0.000    0.000    0.001    0.001 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/resource_tracker.py:70(ensure_running)
       60    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:486(_init_module_attrs)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/bz2.py:1(<module>)
        7    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/enum.py:179(__new__)
       32    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/std.py:451(fp_write)
       66    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/utils.py:194(inner)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/signal.py:1(<module>)
       34    0.000    0.000    0.000    0.000 {method 'write' of '_io.TextIOWrapper' objects}
        2    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:1565(_fill_cache)
      605    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:121(_path_join)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/util.py:447(spawnv_passfds)
        2    0.000    0.000    0.000    0.000 {built-in method posix.listdir}
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/subprocess.py:10(<module>)
        5    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/sre_parse.py:944(parse)
    11829    0.000    0.000    0.000    0.000 {built-in method unicodedata.east_asian_width}
     23/5    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/sre_parse.py:436(_parse_sub)
     25/5    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/sre_parse.py:494(_parse)
        1    0.000    0.000    0.000    0.000 {built-in method _posixsubprocess.fork_exec}
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/pkgutil.py:1(<module>)
      134    0.000    0.000    0.000    0.000 {built-in method __new__ of type object at 0x104813388}
       31    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/abc.py:105(__new__)
        7    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/enum.py:545(<listcomp>)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/struct.py:1(<module>)
       76    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:361(cache_from_source)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/_monitor.py:30(__init__)
        5    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/sre_compile.py:622(_code)
       96    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:385(cached)
       64    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:156(__enter__)
     5915    0.000    0.000    0.000    0.000 {method 'append' of 'list' objects}
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/version.py:1(<module>)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/bisect.py:1(<module>)
      100    0.000    0.000    0.000    0.000 /Users/wenyan/projects2024/efficient_python/test_queue.py:7(close)
      605    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:123(<listcomp>)
       58    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:491(_get_cached)
       86    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/enum.py:88(__setitem__)
     45/5    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/sre_compile.py:87(_compile)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/std.py:245(tqdm)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/importlib/abc.py:1(<module>)
      119    0.000    0.000    0.000    0.000 {method 'format' of 'str' objects}
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/pickle.py:197(<listcomp>)
        2    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/std.py:1265(close)
       74    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:166(_get_module_lock)
        3    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/collections/__init__.py:345(namedtuple)
        1    0.000    0.000    0.000    0.000 {built-in method fromtimestamp}
      107    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/re.py:188(match)
      631    0.000    0.000    0.000    0.000 {built-in method builtins.getattr}
     1858    0.000    0.000    0.000    0.000 {method 'remove' of 'collections.deque' objects}
       58    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:1509(_get_spec)
       38    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:1077(path_stats)
       64    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:160(__exit__)
       35    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/std.py:102(acquire)
       74    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:112(release)
       32    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/std.py:1446(format_dict)
       63    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/std.py:400(format_interval)
       64    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/utils.py:273(_is_ascii)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/numbers.py:4(<module>)
       76    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:127(_path_split)
      181    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/std.py:231(__call__)
        2    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/threading.py:528(__init__)
       35    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/std.py:106(release)
       62    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/_distutils_hack/__init__.py:82(find_spec)
        7    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/enum.py:221(<setcomp>)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/utils.py:34(envwrap)
      695    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:231(_verbose_message)
        2    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/threading.py:228(__init__)
      104    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:398(parent)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/utils.py:73(<dictcomp>)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/threading.py:880(start)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/utils.py:76(wrap)
      143    0.000    0.000    0.000    0.000 {built-in method builtins.max}
      690    0.000    0.000    0.000    0.000 {method 'join' of 'str' objects}
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/inspect.py:3111(signature)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/inspect.py:2859(from_callable)
       58    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:696(spec_from_file_location)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/inspect.py:2246(_signature_from_callable)
       74    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:87(acquire)
       31    0.000    0.000    0.000    0.000 {built-in method now}
       32    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/std.py:186(__format__)
      664    0.000    0.000    0.000    0.000 {built-in method builtins.hasattr}
       12    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/datetime.py:492(__new__)
       35    0.000    0.000    0.000    0.000 {method 'acquire' of '_multiprocessing.SemLock' objects}
        3    0.000    0.000    0.000    0.000 {built-in method builtins.eval}
       44    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/_collections_abc.py:849(__iter__)
     1109    0.000    0.000    0.000    0.000 {method 'startswith' of 'str' objects}
      121    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/enum.py:462(__setattr__)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/threading.py:802(__init__)
       96    0.000    0.000    0.000    0.000 {method 'sub' of 're.Pattern' objects}
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/inspect.py:2152(_signature_from_function)
        2    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/signal.py:67(pthread_sigmask)
       38    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:560(_classify_pyc)
      172    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:1346(_path_importer_cache)
      100    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/threading.py:381(notify_all)
     1286    0.000    0.000    0.000    0.000 {method 'rstrip' of 'str' objects}
      121    0.000    0.000    0.000    0.000 {built-in method builtins.setattr}
      114    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:79(_unpack_uint32)
      244    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:878(__exit__)
        7    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/enum.py:164(__prepare__)
       38    0.000    0.000    0.000    0.000 {method '__exit__' of '_io._IOBase' objects}
        5    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/sre_compile.py:560(_compile_info)
       64    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:185(cb)
      244    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/sre_parse.py:165(__getitem__)
      135    0.000    0.000    0.000    0.000 {built-in method time.time}
      244    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:874(__enter__)
       31    0.000    0.000    0.000    0.000 {built-in method _abc._abc_init}
    54/19    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/sre_parse.py:175(getwidth)
      235    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/socket.py:78(<lambda>)
       38    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:593(_validate_timestamp_pyc)
      237    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/socket.py:88(<lambda>)
      236    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/socket.py:83(<lambda>)
      238    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/socket.py:93(<lambda>)
       64    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:58(__init__)
        3    0.000    0.000    0.000    0.000 {built-in method builtins.dir}
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/threading.py:563(wait)
     1015    0.000    0.000    0.000    0.000 {method 'isupper' of 'str' objects}
       20    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/sre_compile.py:292(_optimize_charset)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/datetime.py:1594(datetime)
       21    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/enum.py:571(_get_mixins_)
       43    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/os.py:674(__getitem__)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/_compat_pickle.py:9(<module>)
      100    0.000    0.000    0.000    0.000 /Users/wenyan/projects2024/efficient_python/test_queue.py:150(append)
        4    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/signal.py:70(<genexpr>)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/synchronize.py:114(_make_name)
       86    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/enum.py:44(_is_private)
       20    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:1179(exec_module)
        3    0.000    0.000    0.000    0.000 {built-in method builtins.next}
      105    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/sre_parse.py:255(get)
       35    0.000    0.000    0.000    0.000 {method 'release' of '_multiprocessing.SemLock' objects}
      233    0.000    0.000    0.000    0.000 {built-in method _thread.allocate_lock}
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/tempfile.py:149(__next__)
       62    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:736(find_spec)
       50    0.000    0.000    0.000    0.000 {method 'update' of 'dict' objects}
        2    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/signal.py:24(_int_to_enum)
       32    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/std.py:153(__init__)
        7    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/importlib/abc.py:20(_register)
       13    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/sre_parse.py:97(closegroup)
        2    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/random.py:117(__init__)
      380    0.000    0.000    0.000    0.000 {method 'rpartition' of 'str' objects}
        2    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/std.py:1286(fp_write)
      152    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:129(<genexpr>)
        2    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/random.py:126(seed)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/util.py:186(__init__)
       28    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/inspect.py:2498(__init__)
      101    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/threading.py:268(_acquire_restore)
      165    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/sre_parse.py:234(__next)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/_compression.py:1(<module>)
       21    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/enum.py:582(_find_data_type)
      101    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/threading.py:265(_release_save)
        2    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:1333(_path_hooks)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/os.py:44(_get_exports_list)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/std.py:686(_decr_instances)
       27    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/abc.py:110(register)
       38    0.000    0.000    0.000    0.000 {built-in method _imp._fix_co_filename}
       10    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:203(_lock_unlock_module)
      695    0.000    0.000    0.000    0.000 {built-in method builtins.ord}
        2    0.000    0.000    0.000    0.000 {function Random.seed at 0x10510e5e0}
      129    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:64(_relax_case)
       69    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/signal.py:9(<lambda>)
       72    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/enum.py:12(_is_descriptor)
        7    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/enum.py:618(_find_new_)
       27    0.000    0.000    0.000    0.000 {built-in method _abc._abc_register}
      382    0.000    0.000    0.000    0.000 {built-in method _imp.release_lock}
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/_monitor.py:1(<module>)
      382    0.000    0.000    0.000    0.000 {built-in method _imp.acquire_lock}
        1    0.000    0.000    0.000    0.000 {built-in method _thread.start_new_thread}
       86    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/enum.py:22(_is_dunder)
      205    0.000    0.000    0.000    0.000 {method 'get' of 'dict' objects}
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/tempfile.py:138(rng)
       86    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/enum.py:33(_is_sunder)
       62    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:351(__init__)
       38    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:523(_check_name_wrapper)
       60    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:811(find_spec)
      104    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/sre_parse.py:250(match)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/utils.py:213(__init__)
        5    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/functools.py:35(update_wrapper)
       27    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/abc.py:121(__subclasscheck__)
       20    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:1155(__init__)
       20    0.000    0.000    0.000    0.000 {built-in method _imp.exec_dynamic}
       86    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/os.py:758(decode)
       43    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/os.py:754(encode)
        2    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:1606(path_hook_for_FileFinder)
       44    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/os.py:697(__iter__)
        7    0.000    0.000    0.000    0.000 {method 'sort' of 'list' objects}
       19    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/sre_parse.py:356(_escape)
      218    0.000    0.000    0.000    0.000 {built-in method builtins.divmod}
       20    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/sre_compile.py:265(_compile_charset)
        3    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/_weakrefset.py:63(__iter__)
        3    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/contextlib.py:234(contextmanager)
        2    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:765(exec_module)
       40    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/enum.py:670(__new__)
       19    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/sre_compile.py:447(_simple)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/std.py:679(_get_free_pos)
       85    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/sre_parse.py:161(__len__)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/process.py:393(__init__)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/gui.py:1(<module>)
      107    0.000    0.000    0.000    0.000 {method 'match' of 're.Pattern' objects}
       45    0.000    0.000    0.000    0.000 {built-in method _imp.is_builtin}
       27    0.000    0.000    0.000    0.000 {built-in method _abc._abc_subclasscheck}
        4    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/std.py:110(__enter__)
       66    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/sre_parse.py:287(tell)
        2    0.000    0.000    0.000    0.000 {built-in method _imp.exec_builtin}
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/inspect.py:2781(__init__)
        5    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/enum.py:977(__and__)
        1    0.000    0.000    0.000    0.000 {built-in method posix.getcwd}
        4    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/std.py:113(__exit__)
        2    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/_weakrefset.py:86(add)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/pickle.py:1136(_Unpickler)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/datetime.py:2216(timezone)
       13    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/sre_parse.py:85(opengroup)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/logging/__init__.py:2034(getLogger)
       38    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:35(_new_module)
        3    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/datetime.py:1602(__new__)
        1    0.000    0.000    0.000    0.000 {built-in method posix.urandom}
       64    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:152(__init__)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/fnmatch.py:1(<module>)
       54    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/sre_parse.py:173(append)
       91    0.000    0.000    0.000    0.000 {built-in method builtins.min}
       78    0.000    0.000    0.000    0.000 {method 'endswith' of 'str' objects}
       34    0.000    0.000    0.000    0.000 {method 'flush' of '_io.TextIOWrapper' objects}
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/tempfile.py:152(<listcomp>)
      149    0.000    0.000    0.000    0.000 {built-in method _thread.get_ident}
        1    0.000    0.000    0.000    0.000 {built-in method fcntl.ioctl}
        2    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:1475(__init__)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/logging/__init__.py:1284(getLogger)
        2    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:1594(<setcomp>)
       45    0.000    0.000    0.000    0.000 {built-in method builtins.locals}
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/threading.py:1229(_make_invoke_excepthook)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/std.py:682(<setcomp>)
       35    0.000    0.000    0.000    0.000 {method 'add' of 'set' objects}
       38    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/__init__.py:23(<genexpr>)
       78    0.000    0.000    0.000    0.000 {method 'rfind' of 'str' objects}
        2    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:757(create_module)
        8    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/random.py:343(choice)
        2    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/datetime.py:665(__neg__)
      130    0.000    0.000    0.000    0.000 {built-in method builtins.abs}
      114    0.000    0.000    0.000    0.000 {built-in method from_bytes}
       22    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:175(_path_isabs)
        1    0.000    0.000    0.000    0.000 {built-in method builtins.print}
       32    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/utils.py:108(__init__)
       45    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/sre_parse.py:112(__init__)
       32    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/std.py:167(colour)
        2    0.000    0.000    0.000    0.000 <frozen zipimport>:63(__init__)
        7    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/enum.py:561(_check_for_existing_members)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/functools.py:799(singledispatch)
       10    0.000    0.000    0.000    0.000 {built-in method builtins.any}
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/_weakrefset.py:111(remove)
      138    0.000    0.000    0.000    0.000 {built-in method posix.fspath}
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/typing.py:1192(__init_subclass__)
        2    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/_weakrefset.py:27(__exit__)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/__init__.py:22(<listcomp>)
        2    0.000    0.000    0.000    0.000 {built-in method _imp.create_builtin}
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/posixpath.py:373(abspath)
      122    0.000    0.000    0.000    0.000 {method 'get' of 'mappingproxy' objects}
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/os.py:48(<listcomp>)
       32    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/utils.py:112(__format__)
       74    0.000    0.000    0.000    0.000 {method 'pop' of 'dict' objects}
        1    0.000    0.000    0.000    0.000 {built-in method posix.write}
       74    0.000    0.000    0.000    0.000 {built-in method builtins.issubclass}
        2    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/functools.py:843(register)
        5    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/datetime.py:415(_check_date_fields)
        5    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/sre_parse.py:225(__init__)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/functools.py:392(__get__)
       37    0.000    0.000    0.000    0.000 {method 'release' of '_thread.RLock' objects}
       60    0.000    0.000    0.000    0.000 {built-in method _imp.is_frozen}
       37    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/sre_parse.py:82(groups)
       63    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/abc.py:7(abstractmethod)
       38    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:1006(__init__)
       86    0.000    0.000    0.000    0.000 {method 'decode' of 'bytes' objects}
        7    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/enum.py:197(<dictcomp>)
        8    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/random.py:237(_randbelow_with_getrandbits)
        2    0.000    0.000    0.000    0.000 {built-in method posix.pipe}
        6    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/sre_parse.py:433(_uniq)
        2    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/datetime.py:1270(__new__)
        5    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/datetime.py:428(_check_time_fields)
       13    0.000    0.000    0.000    0.000 {method 'extend' of 'list' objects}
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/numbers.py:32(Complex)
       91    0.000    0.000    0.000    0.000 {method 'items' of 'dict' objects}
       46    0.000    0.000    0.000    0.000 {method 'encode' of 'str' objects}
        2    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:415(spec_from_loader)
        7    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/enum.py:415(__getattr__)
       72    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/enum.py:551(<lambda>)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/subprocess.py:700(Popen)
        2    0.000    0.000    0.000    0.000 {built-in method atexit.register}
       37    0.000    0.000    0.000    0.000 {method 'acquire' of '_thread.RLock' objects}
        2    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/_weakrefset.py:53(_commit_removals)
       60    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:406(has_location)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/functools.py:517(decorating_function)
       29    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/inspect.py:2830(<genexpr>)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/std.py:131(Bar)
        2    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/datetime.py:823(__new__)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/threading.py:1358(current_thread)
        3    0.000    0.000    0.000    0.000 {method 'remove' of 'set' objects}
       35    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/datetime.py:383(_check_int_field)
       36    0.000    0.000    0.000    0.000 {method 'find' of 'bytearray' objects}
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/sre_compile.py:435(_mk_bitmap)
        5    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/sre_compile.py:485(_get_literal_prefix)
        2    0.000    0.000    0.000    0.000 {built-in method _signal.pthread_sigmask}
        2    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/utils.py:187(disable_on_exception)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/importlib/abc.py:392(Traversable)
        2    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:159(_path_isdir)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/posixpath.py:334(normpath)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/std.py:438(status_printer)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/subprocess.py:282(_args_from_interpreter_flags)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/utils.py:266(_supports_unicode)
       38    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:1031(get_filename)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/util.py:171(register_after_fork)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/datetime.py:793(date)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/process.py:71(BaseProcess)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/context.py:30(BaseContext)
        7    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/enum.py:81(__init__)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/pickle.py:407(_Pickler)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/socket.py:214(socket)
       45    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/_distutils_hack/__init__.py:89(<lambda>)
        3    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/sre_parse.py:268(getuntil)
        5    0.000    0.000    0.000    0.000 {built-in method _sre.compile}
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/weakref.py:105(__init__)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/threading.py:1162(daemon)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/numbers.py:147(Real)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/selectors.py:584(_can_use)
        2    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/_collections_abc.py:925(clear)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/bz2.py:27(BZ2File)
       85    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/_compat_pickle.py:167(<genexpr>)
       21    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/enum.py:244(<genexpr>)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/numbers.py:294(Integral)
       32    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/std.py:163(colour)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/typing.py:1007(__init_subclass__)
       38    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:841(create_module)
        6    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/signal.py:49(decorator)
       44    0.000    0.000    0.000    0.000 {method 'isidentifier' of 'str' objects}
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/std.py:76(TqdmDefaultWriteLock)
       70    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/signal.py:16(<lambda>)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/datetime.py:1245(time)
        5    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/sre_parse.py:928(fix_flags)
       12    0.000    0.000    0.000    0.000 {built-in method builtins.round}
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/utils.py:156(__init__)
       13    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/sre_compile.py:81(_combine_flags)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/logging/__init__.py:1335(_fixupParents)
       10    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/sre_compile.py:619(isstring)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/weakref.py:165(__setitem__)
       71    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/signal.py:21(<lambda>)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/datetime.py:473(timedelta)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/logging/__init__.py:1404(__init__)
       51    0.000    0.000    0.000    0.000 {method 'lower' of 'str' objects}
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/inspect.py:494(unwrap)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/random.py:101(Random)
        3    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/utils.py:152(wrapper_setattr)
       43    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/_compat_pickle.py:165(<genexpr>)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/utils.py:252(_is_utf)
       13    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/collections/__init__.py:419(<genexpr>)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/tempfile.py:643(SpooledTemporaryFile)
       37    0.000    0.000    0.000    0.000 {method 'items' of 'mappingproxy' objects}
        2    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/_weakrefset.py:37(__init__)
        5    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/sre_parse.py:76(__init__)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/sre_compile.py:437(<listcomp>)
        3    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/sre_compile.py:516(_get_charset_prefix)
       19    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/sre_parse.py:169(__setitem__)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/selectors.py:81(BaseSelector)
        2    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/sre_compile.py:456(_generate_overlap_table)
        6    0.000    0.000    0.000    0.000 {built-in method fromkeys}
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/weakref.py:368(__init__)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/sre_parse.py:259(getwhile)
        2    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/weakref.py:496(popitem)
       10    0.000    0.000    0.000    0.000 {method 'setter' of 'property' objects}
        1    0.000    0.000    0.000    0.000 {built-in method posix.sysconf}
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/lzma.py:38(LZMAFile)
       16    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap_external>:1481(<genexpr>)
        2    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/_weakrefset.py:21(__enter__)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/synchronize.py:184(RLock)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/importlib/abc.py:200(InspectLoader)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/utils.py:125(__eq__)
        7    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/enum.py:443(__members__)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/posixpath.py:60(isabs)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/std.py:67(TRLock)
        3    0.000    0.000    0.000    0.000 {built-in method posix.close}
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/context.py:220(Process)
       21    0.000    0.000    0.000    0.000 {method 'pop' of 'set' objects}
        2    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/sre_parse.py:296(_class_escape)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/functools.py:354(__init__)
        3    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/std.py:226(__init__)
       13    0.000    0.000    0.000    0.000 {built-in method sys.intern}
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/reduction.py:251(AbstractReducer)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/synchronize.py:360(Barrier)
        2    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:241(_requires_builtin_wrapper)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/weakref.py:290(update)
        2    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/utils.py:139(__getattr__)
       28    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/inspect.py:2548(name)
        2    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/std.py:1157(__hash__)
        1    0.000    0.000    0.000    0.000 {built-in method math.sqrt}
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/importlib/abc.py:346(ResourceReader)
        3    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/functools.py:65(wraps)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/synchronize.py:210(Condition)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/os.py:804(fsencode)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/socket.py:661(SocketIO)
       10    0.000    0.000    0.000    0.000 {method 'getrandbits' of '_random.Random' objects}
        7    0.000    0.000    0.000    0.000 {built-in method builtins.vars}
       13    0.000    0.000    0.000    0.000 {method '__contains__' of 'frozenset' objects}
        8    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/sre_compile.py:477(_get_iscased)
        2    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/functools.py:840(_is_valid_dispatch_type)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/_tqdm_pandas.py:1(<module>)
        3    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/datetime.py:2236(_create)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/logging/__init__.py:218(_acquireLock)
        2    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/std.py:1153(_comparable)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/random.py:217(__init_subclass__)
        3    0.000    0.000    0.000    0.000 {method 'split' of 'str' objects}
        2    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/utils.py:222(__eq__)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/_compression.py:33(DecompressReader)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/subprocess.py:108(CalledProcessError)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/synchronize.py:46(SemLock)
        4    0.000    0.000    0.000    0.000 {built-in method builtins.id}
        1    0.000    0.000    0.000    0.000 {method 'fileno' of '_io.TextIOWrapper' objects}
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/numbers.py:267(Rational)
        8    0.000    0.000    0.000    0.000 {method 'bit_length' of 'int' objects}
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/tempfile.py:786(TemporaryDirectory)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/_collections_abc.py:779(items)
        5    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/datetime.py:50(_days_in_month)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/reduction.py:33(ForkingPickler)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/selectors.py:291(SelectSelector)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/selectors.py:342(_PollLikeSelector)
        7    0.000    0.000    0.000    0.000 {method 'mro' of 'type' objects}
        3    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/datetime.py:45(_days_before_year)
        4    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/utils.py:17(<genexpr>)
        3    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/utils.py:16(<genexpr>)
        5    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/reduction.py:43(register)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/util.py:108(_platform_supports_abstract_sockets)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/threading.py:757(_newname)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/datetime.py:1214(IsoCalendarDate)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/resource_tracker.py:46(ResourceTracker)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/types.py:171(__get__)
        1    0.000    0.000    0.000    0.000 {built-in method math.exp}
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/util.py:182(Finalize)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/importlib/abc.py:88(PathEntryFinder)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/logging/__init__.py:227(_releaseLock)
        2    0.000    0.000    0.000    0.000 {method 'pop' of 'list' objects}
        2    0.000    0.000    0.000    0.000 {built-in method math.log}
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/pickle.py:200(_Framer)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/resource_tracker.py:48(__init__)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/pkgutil.py:269(ImpLoader)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/importlib/util.py:266(LazyLoader)
        1    0.000    0.000    0.000    0.000 {built-in method builtins.sorted}
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/weakref.py:348(__new__)
        2    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/_weakrefset.py:17(__init__)
        1    0.000    0.000    0.000    0.000 {method 'close' of 'select.kqueue' objects}
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/tempfile.py:128(_RandomNameSequence)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/importlib/abc.py:459(TraversableResources)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/selectors.py:207(_BaseSelectorImpl)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/subprocess.py:145(TimeoutExpired)
        7    0.000    0.000    0.000    0.000 {method 'setdefault' of 'dict' objects}
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/utils.py:183(DisableOnWriteError)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/context.py:226(DefaultContext)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/tempfile.py:427(_TemporaryFileCloser)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/subprocess.py:428(CompletedProcess)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/importlib/abc.py:31(Finder)
        1    0.000    0.000    0.000    0.000 {method 'translate' of 'bytearray' objects}
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/typing.py:1350(runtime_checkable)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/gui.py:24(tqdm_gui)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/synchronize.py:90(_make_methods)
        5    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/datetime.py:445(_check_tzinfo_arg)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/selectors.py:434(PollSelector)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/selectors.py:507(KqueueSelector)
        2    0.000    0.000    0.000    0.000 {method 'popitem' of 'dict' objects}
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/subprocess.py:650(_use_posix_spawn)
        1    0.000    0.000    0.000    0.000 {built-in method posix.register_at_fork}
        2    0.000    0.000    0.000    0.000 {built-in method _weakref.proxy}
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/functools.py:478(lru_cache)
        2    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/inspect.py:159(isfunction)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/utils.py:117(Comparable)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/context.py:203(reducer)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/process.py:358(_ParentProcess)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/selectors.py:61(_SelectorMapping)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/_monitor.py:15(TMonitor)
        2    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/typing.py:1197(<genexpr>)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/pickle.py:263(_Unframer)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/random.py:778(SystemRandom)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/datetime.py:1145(tzinfo)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/utils.py:163(SimpleTextIOWrapper)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/context.py:229(__init__)
        6    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/signal.py:48(_wraps)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/_compression.py:9(BaseStream)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/importlib/abc.py:184(ResourceLoader)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/weakref.py:353(__init__)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/std.py:1147(__del__)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/utils.py:102(FormatReplace)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/importlib/abc.py:253(ExecutionLoader)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/logging/__init__.py:1224(__init__)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/threading.py:82(RLock)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/utils.py:138(ObjectWrapper)
        1    0.000    0.000    0.000    0.000 {method 'difference' of 'set' objects}
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/posixpath.py:41(_get_sep)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/inspect.py:514(_is_wrapper)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/std.py:98(<listcomp>)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/synchronize.py:321(Event)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/tempfile.py:470(_TemporaryFileWrapper)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/runpy.py:25(_TempModule)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/pkgutil.py:195(ImpImporter)
        3    0.000    0.000    0.000    0.000 {built-in method sys._getframe}
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/logging/__init__.py:771(__init__)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/logging/__init__.py:193(_checkLevel)
        2    0.000    0.000    0.000    0.000 {built-in method posix.getpid}
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/numbers.py:12(Number)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/context.py:272(ForkProcess)
        1    0.000    0.000    0.000    0.000 {method 'disable' of '_lsprof.Profiler' objects}
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/context.py:293(ForkContext)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/util.py:368(ForkAwareThreadLock)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/importlib/abc.py:299(SourceLoader)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/std.py:214(EMA)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/context.py:233(get_context)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/importlib/abc.py:51(MetaPathFinder)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/_collections_abc.py:802(__init__)
        1    0.000    0.000    0.000    0.000 /Users/wenyan/projects2024/efficient_python/test_queue.py:147(__init__)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/utils.py:226(CallbackIOWrapper)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/utils.py:282(_screen_shape_wrapper)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/context.py:197(get_start_method)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/context.py:279(SpawnProcess)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/context.py:286(ForkServerProcess)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/context.py:301(ForkServerContext)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/process.py:391(_MainProcess)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/synchronize.py:123(Semaphore)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/runpy.py:48(_ModifiedArgv0)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/importlib/util.py:219(_LazyModule)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/importlib/abc.py:137(Loader)
        3    0.000    0.000    0.000    0.000 {built-in method builtins.globals}
        2    0.000    0.000    0.000    0.000 {method 'replace' of 'str' objects}
        2    0.000    0.000    0.000    0.000 {built-in method builtins.callable}
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/std.py:40(TqdmWarning)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/synchronize.py:142(BoundedSemaphore)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/synchronize.py:159(Lock)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/subprocess.py:272(_optim_args_from_interpreter_flags)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/process.py:343(AuthenticationString)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/reduction.py:211(_C)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/util.py:385(ForkAwareLocal)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/subprocess.py:105(SubprocessError)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/_monitor.py:9(TqdmSynchronisationWarning)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/utils.py:79(<dictcomp>)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/_dist_ver.py:1(<module>)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/context.py:14(ProcessError)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/socket.py:211(_GiveupOnSendfile)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/spawn.py:45(get_executable)
        2    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/threading.py:536(is_set)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/threading.py:1147(daemon)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/pickle.py:97(_Stop)
        2    0.000    0.000    0.000    0.000 <frozen importlib._bootstrap>:782(is_package)
        1    0.000    0.000    0.000    0.000 {method 'insert' of 'list' objects}
        1    0.000    0.000    0.000    0.000 {built-in method sys.getrecursionlimit}
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/enum.py:792(value)
        1    0.000    0.000    0.000    0.000 <string>:1(<module>)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/context.py:20(TimeoutError)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/context.py:23(AuthenticationError)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/context.py:297(SpawnContext)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/pickle.py:73(PickleError)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/pickle.py:77(PicklingError)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/pickle.py:84(UnpicklingError)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/shutil.py:69(Error)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/shutil.py:72(SameFileError)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/shutil.py:79(ExecError)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/util.py:48(debug)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/runpy.py:165(_Error)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/importlib/abc.py:290(FileLoader)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/inspect.py:2865(parameters)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/std.py:36(TqdmKeyError)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/std.py:52(TqdmExperimentalWarning)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/context.py:17(BufferTooShort)
        1    0.000    0.000    0.000    0.000 {method '__exit__' of '_thread.RLock' objects}
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/std.py:32(TqdmTypeError)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/std.py:57(TqdmDeprecationWarning)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/std.py:62(TqdmMonitorWarning)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/site-packages/tqdm/std.py:1301(<lambda>)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/multiprocessing/process.py:37(current_process)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/shutil.py:75(SpecialFileError)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/shutil.py:82(ReadError)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/shutil.py:85(RegistryError)
        1    0.000    0.000    0.000    0.000 /opt/anaconda3/envs/calc_photo/lib/python3.9/shutil.py:89(_GiveupOnFastCopy)
        1    0.000    0.000    0.000    0.000 {method '__init_subclass__' of 'object' objects}

我们把多线程和单线程的profile文件对比来看一下。

从提供的两个 cProfile 性能分析报告来看，第一个报告是多线程版本的程序分析结果，而第二个报告是单线程版本的程序分析结果。下面是两者的对比和性能瓶颈原因分析：

多线程版本性能分析报告（threaded_prof）：

总函数调用次数：18,014,274次。
总运行时间：3.377秒。
主要瓶颈：
- queue.put：450,100次调用，累积耗时1.163秒。
- 线程锁操作（__enter__ 和 __exit__）：1,350,400次调用，累积耗时0.723秒和0.165秒。

单线程版本性能分析报告（threaded_prof_single_thread）：

总函数调用次数：5,871,267次。
总运行时间：1.004秒。
主要瓶颈：
- conway_game.py:step_cell：450,000次调用，累积耗时0.907秒。
- conway_game.py:count_neighbors：450,000次调用，累积耗时0.648秒。

性能瓶颈原因分析：

多线程开销：
- 多线程版本中，大量的时间花费在了队列的 put 和 get 操作以及线程锁的管理上。这表明线程间的同步和通信可能是性能瓶颈。
锁竞争：
- 多线程程序中，线程锁的获取和释放操作非常频繁，这可能导致线程竞争，从而影响性能。
I/O操作：
- 在单线程版本中，print 函数的调用（splitlines）占用了一定的时间，而在多线程版本中，I/O操作的影响相对较小。
算法效率：
- 单线程版本的主要时间花费在 step_cell 和 count_neighbors 函数上，这表明算法本身的效率可能是性能瓶颈。
CPU核心利用率：
- 单线程版本受限于单个CPU核心的处理能力，而多线程版本可能由于线程切换和同步开销，并没有实现预期的加速比。
内存使用：
- 多线程版本可能需要更多的内存来存储线程栈和共享数据结构，这可能影响缓存效率和内存带宽。
程序结构：
- 多线程版本的程序结构可能更复杂，增加了调试和优化的难度。

结论：

从两个报告的对比来看，多线程版本并没有实现预期的性能提升，主要瓶颈在于线程间的同步和通信开销。

bug404_

关注

14
点赞
踩
9

收藏

觉得还不错? 一键收藏
0
评论
python并发与并行（六） ———— 正确的重构代码，以便用Queue做并发

在前面“python并发与并行（五.2） ———— 不要在每次fan-out时都新建一批Thread实例”里面，大家看到，每次都手工创建一批线程并平行地执行I/O任务是有很多缺点的。这一条要介绍另一种方案，也就是用内置的queue模块里的Queue类实现多线程管道。Queue方案的总思路是：在推进游戏时，不像原来那样，每推进一代，就新建一批线程来推进相应的单元格，而是可以提前创建数量固定的一组工作线程，令这组线程平行地处理当前这批I/O任务，并在处理完之后，继续等待下一批任务，这样就不会消耗那么多资源了，
复制链接

扫一扫

专栏目录