java并发编程之源码分析ThreadPoolExecutor线程池实现原理

2401_84433600

已于 2024-04-20 22:41:40 修改

阅读量611

点赞数 29

分类专栏： 2024年程序员学习文章标签： java 服务器性能优化

于 2024-04-20 22:41:38 首次发布

本文链接：https://blog.csdn.net/2401_84433600/article/details/138014152

版权

2024年程序员学习专栏收录该内容

33 篇文章 0 订阅

订阅专栏

先自我介绍一下，小编浙江大学毕业，去过华为、字节跳动等大厂，目前阿里P7

深知大多数程序员，想要提升技能，往往是自己摸索成长，但自己不成体系的自学效果低效又漫长，而且极易碰到天花板技术停滞不前！

因此收集整理了一份《2024年最新Java开发全套学习资料》，初衷也很简单，就是希望能够帮助到想自学提升又不知道该从何学起的朋友。

既有适合小白学习的零基础资料，也有适合3年以上经验的小伙伴深入学习提升的进阶课程，涵盖了95%以上Java开发知识点，真正体系化！

由于文件比较多，这里只是将部分目录截图出来，全套包含大厂面经、学习笔记、源码讲义、实战项目、大纲路线、讲解视频，并且后续会持续更新

如果你需要这些资料，可以添加V获取：vip1024b （备注Java）

正文

```
   pool
```
@param keepAliveTime when the number of threads is greater than

   the core, this is the maximum time that excess idle threads

   will wait for new tasks before terminating.

@param unit the time unit for the {@code keepAliveTime} argument
@param workQueue the queue to use for holding tasks before they are

   executed.  This queue will hold only the {@code Runnable}

   tasks submitted by the {@code execute} method.

@param threadFactory the factory to use when the executor
```
   creates a new thread
```
@param handler the handler to use when execution is blocked

   because the thread bounds and queue capacities are reached

@throws IllegalArgumentException if one of the following holds:
```
    {@code corePoolSize < 0}<br>
```
```
    {@code keepAliveTime < 0}<br>
```
```
    {@code maximumPoolSize <= 0}<br>
```

    {@code maximumPoolSize < corePoolSize}

@throws NullPointerException if {@code workQueue}

    or {@code threadFactory} or {@code handler} is null

public ThreadPoolExecutor(int corePoolSize,

int maximumPoolSize,

long keepAliveTime,

TimeUnit unit,

BlockingQueue workQueue,

ThreadFactory threadFactory,

RejectedExecutionHandler handler) {

if (corePoolSize < 0 ||

maximumPoolSize <= 0 ||

maximumPoolSize < corePoolSize ||

keepAliveTime < 0)

throw new IllegalArgumentException();

if (workQueue == null || threadFactory == null || handler == null)

throw new NullPointerException();

this.corePoolSize = corePoolSize;

this.maximumPoolSize = maximumPoolSize;

this.workQueue = workQueue;

this.keepAliveTime = unit.toNanos(keepAliveTime);

this.threadFactory = threadFactory;

this.handler = handler;

}

corePoolSize 核心线程数
maximumPoolSize 最大线程数
keepAliveTime 线程保持激活状态的时间，如果为0，永远处于激活状态
unit ，keepAliveTime的单位
workQueue，线程池使用的队列
threadFactory 创建线程的工厂
handler 当队列已满，无更大线程处理任务时的拒绝任务的策略。

除了这些核心参数外，我觉得有必要再关注如下。

HashSet workers
completedTaskCount 完成的任务数
allowCoreThreadTimeOut，该值默认为false,也就是默认keepAliveTime不会生效。

3、核心源码分析

========

3.1 线程状态与几个基础方法设计原理

/**

The main pool control state, ctl, is an atomic integer packing
two conceptual fields
workerCount, indicating the effective number of threads
runState, indicating whether running, shutting down etc
In order to pack them into one int, we limit workerCount to
(2^29)-1 (about 500 million) threads rather than (2^31)-1 (2
billion) otherwise representable. If this is ever an issue in
the future, the variable can be changed to be an AtomicLong,
and the shift/mask constants below adjusted. But until the need
arises, this code is a bit faster and simpler using an int.
The workerCount is the number of workers that have been
permitted to start and not permitted to stop. The value may be
transiently different from the actual number of live threads,
for example when a ThreadFactory fails to create a thread when
asked, and when exiting threads are still performing
bookkeeping before terminating. The user-visible pool size is
reported as the current size of the workers set.
The runState provides the main lifecyle control, taking on values:
RUNNING: Accept new tasks and process queued tasks
SHUTDOWN: Don’t accept new tasks, but process queued tasks
STOP: Don’t accept new tasks, don’t process queued tasks,

        and interrupt in-progress tasks

TIDYING: All tasks have terminated, workerCount is zero,

        the thread transitioning to state TIDYING

        will run the terminated() hook method

TERMINATED: terminated() has completed
The numerical order among these values matters, to allow
ordered comparisons. The runState monotonically increases over
time, but need not hit each state. The transitions are:
RUNNING -> SHUTDOWN
On invocation of shutdown(), perhaps implicitly in finalize()
(RUNNING or SHUTDOWN) -> STOP
On invocation of shutdownNow()
SHUTDOWN -> TIDYING
When both queue and pool are empty
STOP -> TIDYING
When pool is empty
TIDYING -> TERMINATED
When the terminated() hook method has completed
Threads waiting in awaitTermination() will return when the
state reaches TERMINATED.
Detecting the transition from SHUTDOWN to TIDYING is less
straightforward than you’d like because the queue may become
empty after non-empty and vice versa during SHUTDOWN state, but
we can only terminate if, after seeing that it is empty, we see
that workerCount is 0 (which sometimes entails a recheck – see
below).

private final AtomicInteger ctl = new AtomicInteger(ctlOf(RUNNING, 0));

private static final int COUNT_BITS = Integer.SIZE - 3;

private static final int CAPACITY = (1 << COUNT_BITS) - 1;

// runState is stored in the high-order bits

private static final int RUNNING = -1 << COUNT_BITS;

private static final int SHUTDOWN = 0 << COUNT_BITS;

private static final int STOP = 1 << COUNT_BITS;

private static final int TIDYING = 2 << COUNT_BITS;

private static final int TERMINATED = 3 << COUNT_BITS;

// Packing and unpacking ctl

private static int runStateOf(int c) { return c & ~CAPACITY; }

private static int workerCountOf(int c) { return c & CAPACITY; }

private static int ctlOf(int rs, int wc) { return rs | wc; }

private static boolean isRunning(int c) {

return c < SHUTDOWN;

}

相关源码解读：

不知大家有没有想过为什么线程池的状态简单的定义为 -1,0,1,2,3 不就得了，为什么还要用移位操作呢？

原来这样的，ThreadPool ctl 的这个变量的设计哲学是用int的高3位 + 29个0代表状态，用高位000+低29位来表示线程池中工作线程的数量，太佩服了。

首先 CAPACITY 的值为 workCount 的最大容量，该值为 000 11111 11111111 11111111 11111111,29个1，

我们来看一下

private static int runStateOf(int c) { return c & ~CAPACITY; }

用ctl里面的值与容量取反的方式获取状态值。由于CAPACITY的值为000 11111 11111111 11111111 11111111,那取反后为111 00000 00000000 00000000 00000000, 用 c 与该值进行与运算，这样就直接保留了c的高三位，然后将c的低29位设置为0,这不就是线程池状态的存放规则吗，绝。

根据此方法，不难得出计算workCount的方法。

private static int ctlOf(int rs, int wc) { return rs | wc; }

该方法，主要是用来更新运行状态的。确保工作线程数量不丢失。

线程池状态以及含义。

RUNNING 运行态
SHUTDOWN 关闭，此时不接受新的任务，但继续处理队列中的任务。
STOP 停止，此时不接受新的任务，不处理队列中的任务，并中断正在执行的任务
TIDYING 所有的工作线程全部停止，并工作线程数量为0，将调用terminated方法，进入到TERMINATED
TERMINATED 终止状态

线程池默认状态 RUNNING

如果调用shutdwon() 方法，状态从 RUNNING —> SHUTDOWN。

如果调用shutdwonNow()方法，状态从RUUNING|SHUTDOWN—>STOP。

SHUTDOWN —> TIDYING 。

队列为空并且线程池空。

STOP --> TIDYING。

线程池为空。

线程池设计原理：

线程池的工作线程为ThreadPoolExecutors的Worker线程，无论是submit还是executor方法中传入的Callable task,Runable参数，只是实现了Runnable接口，在线程池的调用过程，不会调用其start方法，只会调用Worker线程的start方法，然后在Worker线程的run方法中会调用入参的run方法。
众所周知，线程的生命周期在run方法运行结束后（包括异常退出）就结束。要想重复利用线程，我们要确保工作线程Worker的run方法运行在一个无限循环中，然后从任务队列中一个一个获取对象，如果任务队列为空，则阻塞，当然需要提供一些控制，结束无限循环，来销毁线程。在源码 runWorker方法与getTask来实现。

大概的实现思路是如果 getTask 返回null, 则该 worker 线程将被销毁。

那 getTask 在什么情况下会返回 false 呢？

如果线程池的状态为SHUTDOWN并且队列不为空
如果线程池的状态大于STOP
如果当前运行的线程数大于核心线程数，会返回null,已销毁该worker线程

对keepAliveTime的理解，如果allowCoreThreadTimeOut为真，那么keepAliveTime其实就是从任务队列获取任务等待的超时时间，也就是workerQueue.poll(keepALiveTime, TimeUnit.NANOSECONDS)

3.2 FUture submit(Callable task) 方法详解

在看的代码的过程中，只要明白了上述基础方法，后面的代码看起来清晰可见，故，我只列出关键方法，大家可以浏览，应该不难。

/**

Submits a value-returning task for execution and returns a
Future representing the pending results of the task. The
Future’s get method will return the task’s result upon
successful completion.
If you would like to immediately block waiting
for a task, you can use constructions of the form
result = exec.submit(aCallable).get();
Note: The {@link Executors} class includes a set of methods
that can convert some other common closure-like objects,
for example, {@link java.security.PrivilegedAction} to
{@link Callable} form so they can be submitted.
@param task the task to submit
@return a Future representing pending completion of the task
@throws RejectedExecutionException if the task cannot be
```
    scheduled for execution
```
@throws NullPointerException if the task is null

Future submit(Callable task);

提交一个任务，并返回结构到Future,Future就是典型的Future设计模式，就是提交任务到线程池后，返回一个凭证，并直接返回，主线程继续执行，然后当线程池将任务运行完毕后，再将结果填充到凭证中，当主线程调用凭证future的get方法时，如果结果还未填充，则阻塞等待。

现将Callable与Future接口的源代码贴出来，然后重点分析submit方法的实现。

public interface Callable {

/**

Computes a result, or throws an exception if unable to do so.
@return computed result
@throws Exception if unable to compute a result

V call() throws Exception;

}

public interface Future {

/**

Attempts to cancel execution of this task. This attempt will
fail if the task has already completed, has already been cancelled,
or could not be cancelled for some other reason. If successful,
and this task has not started when cancel is called,
this task should never run. If the task has already started,
then the mayInterruptIfRunning parameter determines
whether the thread executing this task should be interrupted in
an attempt to stop the task.
After this method returns, subsequent calls to {@link #isDone} will
always return true. Subsequent calls to {@link #isCancelled}
will always return true if this method returned true.
@param mayInterruptIfRunning true if the thread executing this
task should be interrupted; otherwise, in-progress tasks are allowed
to complete
@return false if the task could not be cancelled,
typically because it has already completed normally;
true otherwise

boolean cancel(boolean mayInterruptIfRunning);

/**

Returns true if this task was cancelled before it completed
normally.
@return true if this task was cancelled before it completed

boolean isCancelled();

/**

Returns true if this task completed.
Completion may be due to normal termination, an exception, or
cancellation – in all of these cases, this method will return
true.
@return true if this task completed

boolean isDone();

/**

Waits if necessary for the computation to complete, and then
retrieves its result.
@return the computed result
@throws CancellationException if the computation was cancelled
@throws ExecutionException if the computation threw an
exception
@throws InterruptedException if the current thread was interrupted
while waiting

V get() throws InterruptedException, ExecutionException;

/**

Waits if necessary for at most the given time for the computation
to complete, and then retrieves its result, if available.
@param timeout the maximum time to wait
@param unit the time unit of the timeout argument
@return the computed result
@throws CancellationException if the computation was cancelled
@throws ExecutionException if the computation threw an
exception
@throws InterruptedException if the current thread was interrupted
while waiting
@throws TimeoutException if the wait timed out

V get(long timeout, TimeUnit unit)

throws InterruptedException, ExecutionException, TimeoutException;

}

现在开始探究submit的实现原理，该代码出自AbstractExecutorService中

public Future<?> submit(Runnable task) {

if (task == null) throw new NullPointerException();

RunnableFuture ftask = newTaskFor(task, null);

execute(ftask);

return ftask;

}

protected RunnableFuture newTaskFor(Callable callable) {

return new FutureTask(callable);

}

核心实现在ThreadPoolExecutor的execute方法

/**

Executes the given task sometime in the future. The task
may execute in a new thread or in an existing pooled thread.
If the task cannot be submitted for execution, either because this
executor has been shutdown or because its capacity has been reached,
the task is handled by the current {@code RejectedExecutionHandler}.
@param command the task to execute
@throws RejectedExecutionException at discretion of

    {@code RejectedExecutionHandler}, if the task

```
    cannot be accepted for execution
```
@throws NullPointerException if {@code command} is null

public void execute(Runnable command) {

if (command == null)

throw new NullPointerException();

Proceed in 3 steps:
1. If fewer than corePoolSize threads are running, try to
start a new thread with the given command as its first
task. The call to addWorker atomically checks runState and
workerCount, and so prevents false alarms that would add
threads when it shouldn’t, by returning false.
1. If a task can be successfully queued, then we still need
to double-check whether we should have added a thread
(because existing ones died since last checking) or that
the pool shut down since entry into this method. So we
recheck state and if necessary roll back the enqueuing if
stopped, or start a new thread if there are none.
1. If we cannot queue task, then we try to add a new
thread. If it fails, we know we are shut down or saturated
and so reject the task.

int c = ctl.get();

if (workerCountOf© < corePoolSize) { // @1

if (addWorker(command, true)) // @2

return;

c = ctl.get(); //@3

}

if (isRunning© && workQueue.offer(command)) {

int recheck = ctl.get();

if (! isRunning(recheck) && remove(command))

reject(command);

else if (workerCountOf(recheck) == 0)

addWorker(null, false);

}

else if (!addWorker(command, false))

reject(command);

}

代码@1,如果当前线程池中的线程数量小于核心线程数的话，尝试新增一个新的线程。所以我们把目光投入到addWorker方法中。

addWorker源码详解：

/**

Checks if a new worker can be added with respect to current
pool state and the given bound (either core or maximum). If so,
the worker count is adjusted accordingly, and, if possible, a
new worker is created and started, running firstTask as its
first task. This method returns false if the pool is stopped or
eligible to shut down. It also returns false if the thread
factory fails to create a thread when asked. If the thread
creation fails, either due to the thread factory returning
null, or due to an exception (typically OutOfMemoryError in
Thread#start), we roll back cleanly.
@param firstTask the task the new thread should run first (or
null if none). Workers are created with an initial first task
(in method execute()) to bypass queuing when there are fewer
than corePoolSize threads (in which case we always start one),
or when the queue is full (in which case we must bypass queue).
Initially idle threads are usually created via
prestartCoreThread or to replace other dying workers.
@param core if true use corePoolSize as bound, else
maximumPoolSize. (A boolean indicator is used here rather than a
value to ensure reads of fresh values after checking other pool
state).
@return true if successful

private boolean addWorker(Runnable firstTask, boolean core) {

retry:

for (;😉 { // @1

int c = ctl.get();

int rs = runStateOf©; // @2

// Check if queue empty only if necessary.

if (rs >= SHUTDOWN && //@3

! (rs == SHUTDOWN &&

firstTask == null &&

! workQueue.isEmpty()))

return false;

for (;😉 { //@4

int wc = workerCountOf©;

if (wc >= CAPACITY ||

wc >= (core ? corePoolSize : maximumPoolSize)) //@5

return false;

if (compareAndIncrementWorkerCount©)

break retry; // @6

c = ctl.get(); // Re-read ctl

if (runStateOf© != rs)

continue retry; //@7

// else CAS failed due to workerCount change; retry inner loop

}

boolean workerStarted = false;

boolean workerAdded = false;

Worker w = null;

try {

final ReentrantLock mainLock = this.mainLock;

w = new Worker(firstTask);

final Thread t = w.thread;

if (t != null) {

mainLock.lock(); // @8

try {

// Recheck while holding lock.

// Back out on ThreadFactory failure or if

// shut down before lock acquired.

int c = ctl.get();

int rs = runStateOf©;

if (rs < SHUTDOWN ||

(rs == SHUTDOWN && firstTask == null)) {

if (t.isAlive()) // precheck that t is startable

throw new IllegalThreadStateException();

workers.add(w);

int s = workers.size();

if (s > largestPoolSize)

largestPoolSize = s;

workerAdded = true;

}

} finally {

mainLock.unlock();

}

if (workerAdded) {

t.start(); // 运行线程 // @9

workerStarted = true;

} //@8 end

}

} finally {

if (! workerStarted)

addWorkerFailed(w); // 增加工作线程失败

}

return workerStarted;

}

代码@1，外层循环（自旋模式）

代码@2，获取线程池的运行状态

代码@3，这里的判断条件，为什么不直接写 if(rs >= SHUTDOWN) return false;而要加第二个条件，目前不明白，等在了解到参数firstTask在什么情况下为空。在这里，我们目前只要知道，只有线程池的状态为 RUNNING时，线程池才接收新的任务，去增加工作线程。

代码@4，内层循环，主要的目的就是利用CAS增加一个线程数量。

代码@5，判断当前线程池的数量，如果数量达到规定的数量，则直接返回false,添加工作线程失败。

代码@6，如果修改线程数量成功，则跳出循环，开始创建工作线程。

代码@7，如果修改线程数量不成功(CAS)有两种情况：1、线程数量变化，重试则好，2，如果线程的运行状态变化，则重新开始外层循环，重新判断addWork流程。

代码@8，在锁mainLock的保护下，完成 workers （HashSet）的维护。

接着分析一下代码@9，启动线程，执行关键的方法 runWorker方法：

/**

Main worker run loop. Repeatedly gets tasks from queue and
executes them, while coping with a number of issues:
1. We may start out with an initial task, in which case we
don’t need to get the first one. Otherwise, as long as pool is
running, we get tasks from getTask. If it returns null then the
worker exits due to changed pool state or configuration
parameters. Other exits result from exception throws in
external code, in which case completedAbruptly holds, which
usually leads processWorkerExit to replace this thread.
1. Before running any task, the lock is acquired to prevent
other pool interrupts while the task is executing, and
clearInterruptsForTaskRun called to ensure that unless pool is
stopping, this thread does not have its interrupt set.
1. Each task run is preceded by a call to beforeExecute, which
might throw an exception, in which case we cause thread to die
(breaking loop with completedAbruptly true) without processing
the task.
1. Assuming beforeExecute completes normally, we run the task,
gathering any of its thrown exceptions to send to
afterExecute. We separately handle RuntimeException, Error
(both of which the specs guarantee that we trap) and arbitrary
Throwables. Because we cannot rethrow Throwables within
Runnable.run, we wrap them within Errors on the way out (to the
thread’s UncaughtExceptionHandler). Any thrown exception also
conservatively causes thread to die.
1. After task.run completes, we call afterExecute, which may
also throw an exception, which will also cause thread to
die. According to JLS Sec 14.20, this exception is the one that
will be in effect even if task.run throws.
The net effect of the exception mechanics is that afterExecute
and the thread’s UncaughtExceptionHandler have as accurate
information as we can provide about any problems encountered by
user code.
@param w the worker

final void runWorker(Worker w) {

Thread wt = Thread.currentThread();

Runnable task = w.firstTask;

w.firstTask = null;

w.unlock(); // allow interrupts

boolean completedAbruptly = true;

try {

while (task != null || (task = getTask()) != null) {

w.lock();

// If pool is stopping, ensure thread is interrupted;

// if not, ensure thread is not interrupted. This

// requires a recheck in second case to deal with

// shutdownNow race while clearing interrupt

if ((runStateAtLeast(ctl.get(), STOP) ||

(Thread.interrupted() &&

runStateAtLeast(ctl.get(), STOP))) &&

!wt.isInterrupted())

wt.interrupt();

try {

beforeExecute(wt, task);

Throwable thrown = null;

try {

task.run();

} catch (RuntimeException x) {

thrown = x; throw x;

} catch (Error x) {

thrown = x; throw x;

} catch (Throwable x) {

thrown = x; throw new Error(x);

} finally {

afterExecute(task, thrown);

}

} finally {

task = null;

w.completedTasks++;

w.unlock();

}

completedAbruptly = false;

} finally {

processWorkerExit(w, completedAbruptly);

}

/**

Performs blocking or timed wait for a task, depending on
current configuration settings, or returns null if this worker
must exit because of any of:
1. There are more than maximumPoolSize workers (due to
a call to setMaximumPoolSize).
1. The pool is stopped.
1. The pool is shutdown and the queue is empty.
1. This worker timed out waiting for a task, and timed-out
workers are subject to termination (that is,
{@code allowCoreThreadTimeOut || workerCount > corePoolSize})
both before and after the timed wait.
@return task, or null if the worker must exit, in which case

总结

这份面试题几乎包含了他在一年内遇到的所有面试题以及答案，甚至包括面试中的细节对话以及语录，可谓是细节到极致，甚至简历优化和怎么投简历更容易得到面试机会也包括在内！也包括教你怎么去获得一些大厂，比如阿里，腾讯的内推名额！

某位名人说过成功是靠99%的汗水和1%的机遇得到的，而你想获得那1%的机遇你首先就得付出99%的汗水！你只有朝着你的目标一步一步坚持不懈的走下去你才能有机会获得成功！

成功只会留给那些有准备的人！

网上学习资料一大堆，但如果学到的知识不成体系，遇到问题时只是浅尝辄止，不再深入研究，那么很难做到真正的技术提升。

需要这份系统化的资料的朋友，可以添加V获取：vip1024b （备注Java）

一个人可以走的很快，但一群人才能走的更远！不论你是正从事IT行业的老鸟或是对IT行业感兴趣的新人，都欢迎加入我们的的圈子（技术交流、学习资源、职场吐槽、大厂内推、面试辅导），让我们一起学习成长！

ptly);

}

/**

Performs blocking or timed wait for a task, depending on
current configuration settings, or returns null if this worker
must exit because of any of:
1. There are more than maximumPoolSize workers (due to
a call to setMaximumPoolSize).
1. The pool is stopped.
1. The pool is shutdown and the queue is empty.
1. This worker timed out waiting for a task, and timed-out
workers are subject to termination (that is,
{@code allowCoreThreadTimeOut || workerCount > corePoolSize})
both before and after the timed wait.
@return task, or null if the worker must exit, in which case

总结

这份面试题几乎包含了他在一年内遇到的所有面试题以及答案，甚至包括面试中的细节对话以及语录，可谓是细节到极致，甚至简历优化和怎么投简历更容易得到面试机会也包括在内！也包括教你怎么去获得一些大厂，比如阿里，腾讯的内推名额！

成功只会留给那些有准备的人！

[外链图片转存中…(img-WPrBmZFU-1713624082146)]

网上学习资料一大堆，但如果学到的知识不成体系，遇到问题时只是浅尝辄止，不再深入研究，那么很难做到真正的技术提升。

需要这份系统化的资料的朋友，可以添加V获取：vip1024b （备注Java）
[外链图片转存中…(img-Z69t6Zsy-1713624082147)]

2401_84433600

关注

29
点赞
踩
9

收藏

觉得还不错? 一键收藏
0
评论
java并发编程之源码分析ThreadPoolExecutor线程池实现原理

这份面试题几乎包含了他在一年内遇到的所有面试题以及答案，甚至包括面试中的细节对话以及语录，可谓是细节到极致，甚至简历优化和怎么投简历更容易得到面试机会也包括在内！也包括教你怎么去获得一些大厂，比如阿里，腾讯的内推名额！某位名人说过成功是靠99%的汗水和1%的机遇得到的，而你想获得那1%的机遇你首先就得付出99%的汗水！你只有朝着你的目标一步一步坚持不懈的走下去你才能有机会获得成功！成功只会留给那些有准备的人！
复制链接

扫一扫