Python3 ThreadPoolExecutor--线程池

1、线程池创建

    def __init__(self, 
        max_workers=None, 
        thread_name_prefix='', 
        initializer=None, 
        initargs=())

max_workers: 设置线程池中最多能同时运行的线程数目

thread_name_prefix：一个可选的名称前缀，用于给线程命名

initializer : 线程可回调的函数

initargs: initializer 的参数

2、submit

提交线程需要执行的任务（函数名和参数）到线程池中，并返回该任务的句柄

def submit(self, fn, /, *args, **kwargs):

即将fn函数提交给线程池，*args代表传给fn函数的参数，**kwargs代表以关键字参数的形式为fn函数传入参数。

    def submit(self, fn, /, *args, **kwargs):
        with self._shutdown_lock, _global_shutdown_lock:
            if self._broken:
                raise BrokenThreadPool(self._broken)

            if self._shutdown:
                raise RuntimeError('cannot schedule new futures after shutdown')
            if _shutdown:
                raise RuntimeError('cannot schedule new futures after '
                                   'interpreter shutdown')

            f = _base.Future()
            w = _WorkItem(f, fn, args, kwargs)

            self._work_queue.put(w)
            self._adjust_thread_count()
            return f

submit 函数返回的为 future 对象 ，future 各个方法如下：

class Future(Generic[_T]):
    # 初始化
    def __init__(self) -> None: ...

    # Returns True if the future was cancelled, False otherwise. A future
        cannot be cancelled if it is running or has already completed.
    def cancel(self) -> bool: ...

    # Return True if the future was cancelled.
    def cancelled(self) -> bool: ...

    # Return True if the future is currently executing
    def running(self) -> bool: ...

    # Return True of the future was cancelled or finished executing
    def done(self) -> bool: ...

    #"""Attaches a callable that will be called when the future finishes.

        Args:
            fn: A callable that will be called with this future as its only
                argument when the future completes or is cancelled. The callable
                will always be called by a thread in the same process in which
                it was added. If the future has already completed or been
                cancelled then the callable will be called immediately. These
                callables are called in the order that they were added.
     """
    def add_done_callback(self, fn: Callable[[Future[_T]], Any]) -> None: ...

    #"""Return the result of the call that the future represents.

        Args:
            timeout: The number of seconds to wait for the result if the future
                isn't done. If None, then there is no limit on the wait time.

        Returns:
            The result of the call that the future represents.

        Raises:
            CancelledError: If the future was cancelled.
            TimeoutError: If the future didn't finish executing before the given
                timeout.
            Exception: If the call raised then that exception will be raised.
        阻塞式的获取任务的返回值
    """
    def result(self, timeout: float | None = ...) -> _T: ...

    #"""Mark the future as running or process any cancel notifications.

        Should only be used by Executor implementations and unit tests.

        If the future has been cancelled (cancel() was called and returned
        True) then any threads waiting on the future completing (though calls
        to as_completed() or wait()) are notified and False is returned.

        If the future was not cancelled then it is put in the running state
        (future calls to running() will return True) and True is returned.

        This method should be called by Executor implementations before
        executing the work associated with this future. If this method returns
        False then the work should not be executed.

        Returns:
            False if the Future was cancelled, True otherwise.

        Raises:
            RuntimeError: if this method was already called or if set_result()
                or set_exception() was called.
     """
    def set_running_or_notify_cancel(self) -> bool: ...

    #"""Sets the return value of work associated with the future.

        Should only be used by Executor implementations and unit tests.
    """
    def set_result(self, result: _T) -> None: ...

    #"""Return the exception raised by the call that the future represents.

        Args:
            timeout: The number of seconds to wait for the exception if the
                future isn't done. If None, then there is no limit on the wait
                time.

        Returns:
            The exception raised by the call that the future represents or None
            if the call completed without raising.

        Raises:
            CancelledError: If the future was cancelled.
            TimeoutError: If the future didn't finish executing before the given
                timeout.
    """
    def exception(self, timeout: float | None = ...) -> BaseException | None: ...

    #"""Return the exception raised by the call that the future represents.

        Args:
            timeout: The number of seconds to wait for the exception if the
                future isn't done. If None, then there is no limit on the wait
                time.

        Returns:
            The exception raised by the call that the future represents or None
            if the call completed without raising.

        Raises:
            CancelledError: If the future was cancelled.
            TimeoutError: If the future didn't finish executing before the given
                timeout.
    """
    def set_exception(self, exception: BaseException | None) -> None: ...

3、as_completed

一次取出所有任务的结果,详情请看 as_completed 源码,tasks 为 submit 返回的 future 列表

from concurrent.futures import ThreadPoolExecutor, as_completed
 
for future in as_completed(tasks):
    data = future.result()

4、map

另一种获取线程池返回结果的方法

    def map(self, fn, *iterables, timeout=None, chunksize=1):
        """Returns an iterator equivalent to map(fn, iter).

        Args:
            fn: A callable that will take as many arguments as there are
                passed iterables.
            timeout: The maximum number of seconds to wait. If None, then there
                is no limit on the wait time.
            chunksize: The size of the chunks the iterable will be broken into
                before being passed to a child process. This argument is only
                used by ProcessPoolExecutor; it is ignored by
                ThreadPoolExecutor.

        Returns:
            An iterator equivalent to: map(func, *iterables) but the calls may
            be evaluated out-of-order.

        Raises:
            TimeoutError: If the entire result iterator could not be generated
                before the given timeout.
            Exception: If fn(*args) raises for any values.
        """

5、wait

方法可以让主线程阻塞，直到满足设定的要求。tasks 为 submit 返回的 future 列表

from concurrent.futures import ThreadPoolExecutor, wait, ALL_COMPLETED, FIRST_COMPLETED

wait(tasks, return_when=ALL_COMPLETED)

def wait(fs, timeout=None, return_when=ALL_COMPLETED):
    """Wait for the futures in the given sequence to complete.

    Args:
        fs: The sequence of Futures (possibly created by different Executors) to
            wait upon.
        timeout: The maximum number of seconds to wait. If None, then there
            is no limit on the wait time.
        return_when: Indicates when this function should return. The options
            are:

            FIRST_COMPLETED - Return when any future finishes or is
                              cancelled.
            FIRST_EXCEPTION - Return when any future finishes by raising an
                              exception. If no future raises an exception
                              then it is equivalent to ALL_COMPLETED.
            ALL_COMPLETED -   Return when all futures finish or are cancelled.

    Returns:
        A named 2-tuple of sets. The first set, named 'done', contains the
        futures that completed (is finished or cancelled) before the wait
        completed. The second set, named 'not_done', contains uncompleted
        futures. Duplicate futures given to *fs* are removed and will be 
        returned only once.
    """