1、ThreadPoolExecutor概述
由于本人英语水平不高,为了不误导大家,我将源码中的注释复制下来,我不翻译原文,我从入学6个视角试图窥探一下ThreadPoolExecutor全貌。
- 创建线程池的方式
- 核心线程数、最大线程数
- 线程的创建
- 线程的Keep-alive(保持存活的空闲时间)
- 队列
- 任务的丢弃策略
/**
* An {@link ExecutorService} that executes each submitted task using
* one of possibly several pooled threads, normally configured
* using {@link Executors} factory methods.
*
* <p>Thread pools address two different problems: they usually
* provide improved performance when executing large numbers of
* asynchronous tasks, due to reduced per-task invocation overhead,
* and they provide a means of bounding and managing the resources,
* including threads, consumed when executing a collection of tasks.
* Each {@code ThreadPoolExecutor} also maintains some basic
* statistics, such as the number of completed tasks.
*
* <p>To be useful across a wide range of contexts, this class
* provides many adjustable parameters and extensibility
* hooks. However, programmers are urged to use the more convenient
* {@link Executors} factory methods {@link
* Executors#newCachedThreadPool} (unbounded thread pool, with
* automatic thread reclamation), {@link Executors#newFixedThreadPool}
* (fixed size thread pool) and {@link
* Executors#newSingleThreadExecutor} (single background thread), that
* preconfigure settings for the most common usage
* scenarios. Otherwise, use the following guide when manually
* configuring and tuning this class:
*
1、提供如下工厂方法创建线程池对象(ExecutorService实现类)
1)Executors.newCachedThreadPool,创建一个线程容量为Integer.MAX_VALUE的线程池,空闲时间为60s。
2)Executors.newFixedThreadPool,创建一个固定容量的线程池
3)Executors.newSingleThreadExecutor,创建一个线程的线程池
* <dl>
*核心线程与最大线程篇
* <dt>Core and maximum pool sizes</dt>
corePoolSize 核心线程数
maximumPoolSize 核心线程数
当一个任务提交到线程池
1) 如果当前线程池中的线程数小于corePoolSize时,直接创建一个先的线程。
2) 如果当前线程池中的线程数大于等于corePoolSize时,如果队列未满,直接将线程放入队列中,不新建线程。
3) 如果队列已满,但线程没有超过maximumPoolSize,则新建一个线程。
在运行过程中,可以通过调用setCorePoolSize,setMaximumPoolSize改变这两个参数
*
* <dd>A {@code ThreadPoolExecutor} will automatically adjust the
* pool size (see {@link #getPoolSize})
* according to the bounds set by
* corePoolSize (see {@link #getCorePoolSize}) and
* maximumPoolSize (see {@link #getMaximumPoolSize}).
*
* When a new task is submitted in method {@link #execute}, and fewer
* than corePoolSize threads are running, a new thread is created to
* handle the request, even if other worker threads are idle. If
* there are more than corePoolSize but less than maximumPoolSize
* threads running, a new thread will be created only if the queue is
* full. By setting corePoolSize and maximumPoolSize the same, you
* create a fixed-size thread pool. By setting maximumPoolSize to an
* essentially unbounded value such as {@code Integer.MAX_VALUE}, you
* allow the pool to accommodate an arbitrary number of concurrent
* tasks. Most typically, core and maximum pool sizes are set only
* upon construction, but they may also be changed dynamically using
* {@link #setCorePoolSize} and {@link #setMaximumPoolSize}. </dd>
*
*
*
* <dt>On-demand construction</dt
* 核心线程的创建通常是有任务提交时新建的,当然,我们可以通过调用prestartCoreThread,或
prestartAllCoreThreads方法,预先创建核心线程数。
* <dd> By default, even core threads are initially created and
* started only when new tasks arrive, but this can be overridden
* dynamically using method {@link #prestartCoreThread} or {@link
* #prestartAllCoreThreads}. You probably want to prestart threads if
* you construct the pool with a non-empty queue. </dd>
*
* <dt>Creating new threads</dt>
*线程创建篇,新线程的创建,默认使用Executors.defautThreadFactory来创建线程,同一个线程创建工厂创建的线程具有相同的线程组,优先级,是否是后台线程(daemon),我们可以提供资金的线程创建工厂来改变这些属性,一般我们使用自己定义的线程工厂,主要的目的还是修改线程的名称,方便理解与跟踪。
* <dd>New threads are created using a {@link ThreadFactory}. If not
* otherwise specified, a {@link Executors#defaultThreadFactory} is
* used, that creates threads to all be in the same {@link
* ThreadGroup} and with the same {@code NORM_PRIORITY} priority and
* non-daemon status. By supplying a different ThreadFactory, you can
* alter the thread's name, thread group, priority, daemon status,
* etc. If a {@code ThreadFactory} fails to create a thread when asked
* by returning null from {@code newThread}, the executor will
* continue, but might not be able to execute any tasks. Threads
* should possess the "modifyThread" {@code RuntimePermission}. If
* worker threads or other threads using the pool do not possess this
* permission, service may be degraded: configuration changes may not
* take effect in a timely manner, and a shutdown pool may remain in a
* state in which termination is possible but not completed.</dd>
*
* <dt>Keep-alive times</dt>
* 如果线程池中线程数量超过了核心线程数,超过的线程如果空闲时间超过了keepAliveTime的线程会被终止;
先提出一个疑问:如果核心线程数设置为10,目前有12个线程,其中有3个超过了keepALiveTime,那有3个线程会被终止,还是只有两个,按照上述描述,应该是2个会被终止,,因为有个管家子 excess threads,从源码中去找答案吧。
* <dd>If the pool currently has more than corePoolSize threads,
* excess threads will be terminated if they have been idle for more
* than the keepAliveTime (see {@link #getKeepAliveTime}). This
* provides a means of reducing resource consumption when the pool is
* not being actively used. If the pool becomes more active later, new
* threads will be constructed. This parameter can also be changed
* dynamically using method {@link #setKeepAliveTime}. Using a value
* of {@code Long.MAX_VALUE} {@link TimeUnit#NANOSECONDS} effectively
* disables idle threads from ever terminating prior to shut down. By
* default, the keep-alive policy applies only when there are more
* than corePoolSizeThreads. But method {@link
* #allowCoreThreadTimeOut(boolean)} can be used to apply this
* time-out policy to core threads as well, so long as the
* keepAliveTime value is non-zero. </dd>
*
* <dt>Queuing</dt>
*
* <dd>Any {@link BlockingQueue} may be used to transfer and hold
* submitted tasks. The use of this queue interacts with pool sizing:
*
* <ul>
*
* <li> If fewer than corePoolSize threads are running, the Executor
* always prefers adding a new thread
* rather than queuing.</li>
*
* <li> If corePoolSize or more threads are running, the Executor
* always prefers queuing a request rather than adding a new
* thread.</li>
*
* <li> If a request cannot be queued, a new thread is created unless
* this would exceed maximumPoolSize, in which case, the task will be
* rejected.</li>
*
* </ul>
*
* There are three general strategies for queuing:
三种队列方案
1)直接传递,所有提交任务任务不入队列,直接传递给线程池。
2)有界队列
3)无界队列
采取何种队列,会对线程池中 核心线程数产生影响
再重复一下 核心线程的产生过程
1)如果当前线程池中线程数小于核心线程数,新任务到达,不管有没有队列,都是直接新建一个核心线程。
2)如果线程池中允许的线程达到核心线程数量时,根据不同的队列机制,有如下的处理方法:
a、如果是直接传递,则直接新增线程运行(没有达到最大线程数量)
b、如果是有界队列,先将任务入队列,如果任务队列已满,在线程数没有超过最大线程数限制的情况下,新
建一个线程来运行任务。
c、无界队列,则线程池中最大的线程数量等于核心线程数量,最大线程数量不会有产生任何影响。
* <ol>
*
* <li> <em> Direct handoffs.</em> A good default choice for a work
* queue is a {@link SynchronousQueue} that hands off tasks to threads
* without otherwise holding them. Here, an attempt to queue a task
* will fail if no threads are immediately available to run it, so a
* new thread will be constructed. This policy avoids lockups when
* handling sets of requests that might have internal dependencies.
* Direct handoffs generally require unbounded maximumPoolSizes to
* avoid rejection of new submitted tasks. This in turn admits the
* possibility of unbounded thread growth when commands continue to
* arrive on average faster than they can be processed. </li>
*
* <li><em> Unbounded queues.</em> Using an unbounded queue (for
* example a {@link LinkedBlockingQueue} without a predefined
* capacity) will cause new tasks to wait in the queue when all
* corePoolSize threads are busy. Thus, no more than corePoolSize
* threads will ever be created. (And the value of the maximumPoolSize
* therefore doesn't have any effect.) This may be appropriate when
* each task is completely independent of others, so tasks cannot
* affect each others execution; for example, in a web page server.
* While this style of queuing can be useful in smoothing out
* transient bursts of requests, it admits the possibility of
* unbounded work queue growth when commands continue to arrive on
* average faster than they can be processed. </li>
*
* <li><em>Bounded queues.</em> A bounded queue (for example, an
* {@link ArrayBlockingQueue}) helps prevent resource exhaustion when
* used with finite maximumPoolSizes, but can be more difficult to
* tune and control. Queue sizes and maximum pool sizes may be traded
* off for each other: Using large queues and small pools minimizes
* CPU usage, OS resources, and context-switching overhead, but can
* lead to artificially low throughput. If tasks frequently block (for
* example if they are I/O bound), a system may be able to schedule
* time for more threads than you otherwise allow. Use of small queues
* generally requires larger pool sizes, which keeps CPUs busier but
* may encounter unacceptable scheduling overhead, which also
* decreases throughput. </li>
*
* </ol>
*
* </dd>
*
* <dt>Rejected tasks</dt>
* 任务拒绝策略
1)AbortPolicy,抛出运行时异常
2)CallerRunsPolicy 调用者直接运行,不在线程中运行。
3)DiscardPolicy 直接将任务丢弃
4)DiscardOldestPolicy 丢弃队列中头部的任务。
* <dd> New tasks submitted in method {@link #execute} will be
* <em>rejected</em> when the Executor has been shut down, and also
* when the Executor uses finite bounds for both maximum threads and
* work queue capacity, and is saturated. In either case, the {@code
* execute} method invokes the {@link
* RejectedExecutionHandler#rejectedExecution} method of its {@link
* RejectedExecutionHandler}. Four predefined handler policies are
* provided:
*
* <ol>
*
* <li> In the default {@link ThreadPoolExecutor.AbortPolicy}, the
* handler throws a runtime {@link RejectedExecutionException} upon
* rejection. </li>
*
* <li> In {@link ThreadPoolExecutor.CallerRunsPolicy}, the thread
* that invokes {@code execute} itself runs the task. This provides a
* simple feedback control mechanism that will slow down the rate that
* new tasks are submitted. </li>
*
* <li> In {@link ThreadPoolExecutor.DiscardPolicy}, a task that
* cannot be executed is simply dropped. </li>
*
* <li>In {@link ThreadPoolExecutor.DiscardOldestPolicy}, if the
* executor is not shut down, the task at the head of the work queue
* is dropped, and then execution is retried (which can fail again,
* causing this to be repeated.) </li>
*
* </ol>
*
* It is possible to define and use other kinds of {@link
* RejectedExecutionHandler} classes. Doing so requires some care
* especially when policies are designed to work only under particular
* capacity or queuing policies. </dd>
*
* <dt>Hook methods</dt>
*
* <dd>This class provides {@code protected} overridable {@link
* #beforeExecute} and {@link #afterExecute} methods that are called
* before and after execution of each task. These can be used to
* manipulate the execution environment; for example, reinitializing
* ThreadLocals, gathering statistics, or adding log
* entries. Additionally, method {@link #terminated} can be overridden
* to perform any special processing that needs to be done once the
* Executor has fully terminated.
*
* <p>If hook or callback methods throw exceptions, internal worker
* threads may in turn fail and abruptly terminate.</dd>
*
* <dt>Queue maintenance</dt>
*
* <dd> Method {@link #getQueue} allows access to the work queue for
* purposes of monitoring and debugging. Use of this method for any
* other purpose is strongly discouraged. Two supplied methods,
* {@link #remove} and {@link #purge} are available to assist in
* storage reclamation when large numbers of queued tasks become
* cancelled.</dd>
*
* <dt>Finalization</dt>
*
* <dd> A pool that is no longer referenced in a program <em>AND</em>
* has no remaining threads will be {@code shutdown} automatically. If
* you would like to ensure that unreferenced pools are reclaimed even
* if users forget to call {@link #shutdown}, then you must arrange
* that unused threads eventually die, by setting appropriate
* keep-alive times, using a lower bound of zero core threads and/or
* setting {@link #allowCoreThreadTimeOut(boolean)}. </dd>
*
* </dl>
*
* <p> <b>Extension example</b>. Most extensions of this class
* override one or more of the protected hook methods. For example,
* here is a subclass that adds a simple pause/resume feature:
*
* <pre> {@code
* class PausableThreadPoolExecutor extends ThreadPoolExecutor {
* private boolean isPaused;
* private ReentrantLock pauseLock = new ReentrantLock();
* private Condition unpaused = pauseLock.newCondition();
*
* public PausableThreadPoolExecutor(...) { super(...); }
*
* protected void beforeExecute(Thread t, Runnable r) {
* super.beforeExecute(t, r);
* pauseLock.lock();
* try {
* while (isPaused) unpaused.await();
* } catch (InterruptedException ie) {
* t.interrupt();
* } finally {
* pauseLock.unlock();
* }
* }
*
* public void pause() {
* pauseLock.lock();
* try {
* isPaused = true;
* } finally {
* pauseLock.unlock();
* }
* }
*
* public void resume() {
* pauseLock.lock();
* try {
* isPaused = false;
* unpaused.signalAll();
* } finally {
* pauseLock.unlock();
* }
* }
* }}</pre>
*
* @since 1.5
* @author Doug Lea
*/
2、ThreadPoolExecutors 内部数据结构与构造方法详解
ThreadPoolExecutors的完整构造函数如下,从构造函数中能得出线程池最核心的属性。
/**
* Creates a new {@code ThreadPoolExecutor} with the given initial
* parameters.
*
* @param corePoolSize the number of threads to keep in the pool, even
* if they are idle, unless {@code allowCoreThreadTimeOut} is set
* @param maximumPoolSize the maximum number of threads to allow in the
* pool
* @param keepAliveTime when the number of threads is greater than
* the core, this is the maximum time that excess idle threads
* will wait for new tasks before terminating.
* @param unit the time unit for the {@code keepAliveTime} argument
* @param workQueue the queue to use for holding tasks before they are
* executed. This queue will hold only the {@code Runnable}
* tasks submitted by the {@code execute} method.
* @param threadFactory the factory to use when the executor
* creates a new thread
* @param handler the handler to use when execution is blocked
* because the thread bounds and queue capacities are reached
* @throws IllegalArgumentException if one of the following holds:<br>
* {@code corePoolSize < 0}<br>
* {@code keepAliveTime < 0}<br>
* {@code maximumPoolSize <= 0}<br>
* {@code maximumPoolSize < corePoolSize}
* @throws NullPointerException if {@code workQueue}
* or {@code threadFactory} or {@code handler} is null
*/
public ThreadPoolExecutor(int corePoolSize,
int maximumPoolSize,
long keepAliveTime,
TimeUnit unit,
BlockingQueue<Runnable> workQueue,
ThreadFactory threadFactory,
RejectedExecutionHandler handler) {
if (corePoolSize < 0 ||
maximumPoolSize <= 0 ||
maximumPoolSize < corePoolSize ||
keepAliveTime < 0)
throw new IllegalArgumentException();
if (workQueue == null || threadFactory == null || handler == null)
throw new NullPointerException();
this.corePoolSize = corePoolSize;
this.maximumPoolSize = maximumPoolSize;
this.workQueue = workQueue;
this.keepAliveTime = unit.toNanos(keepAliveTime);
this.threadFactory = threadFactory;
this.handler = handler;
}
- corePoolSize 核心线程数
- maximumPoolSize 最大线程数
- keepAliveTime 线程保持激活状态的时间,如果为0,永远处于激活状态
- unit ,keepAliveTime的单位
- workQueue,线程池使用的队列
- threadFactory 创建线程的工厂
- handler 当队列已满,无更大线程处理任务时的拒绝任务的策略。
除了这些核心参数外,我觉得有必要再关注如下。
- HashSet<Worker> workers
- completedTaskCount 完成的任务数
- allowCoreThreadTimeOut,该值默认为false,也就是默认keepAliveTime不会生效。
3、核心源码分析
3.1 线程状态与几个基础方法设计原理
/**
* The main pool control state, ctl, is an atomic integer packing
* two conceptual fields
* workerCount, indicating the effective number of threads
* runState, indicating whether running, shutting down etc
*
* In order to pack them into one int, we limit workerCount to
* (2^29)-1 (about 500 million) threads rather than (2^31)-1 (2
* billion) otherwise representable. If this is ever an issue in
* the future, the variable can be changed to be an AtomicLong,
* and the shift/mask constants below adjusted. But until the need
* arises, this code is a bit faster and simpler using an int.
*
* The workerCount is the number of workers that have been
* permitted to start and not permitted to stop. The value may be
* transiently different from the actual number of live threads,
* for example when a ThreadFactory fails to create a thread when
* asked, and when exiting threads are still performing
* bookkeeping before terminating. The user-visible pool size is
* reported as the current size of the workers set.
*
* The runState provides the main lifecyle control, taking on values:
*
* RUNNING: Accept new tasks and process queued tasks
* SHUTDOWN: Don't accept new tasks, but process queued tasks
* STOP: Don't accept new tasks, don't process queued tasks,
* and interrupt in-progress tasks
* TIDYING: All tasks have terminated, workerCount is zero,
* the thread transitioning to state TIDYING
* will run the terminated() hook method
* TERMINATED: terminated() has completed
*
* The numerical order among these values matters, to allow
* ordered comparisons. The runState monotonically increases over
* time, but need not hit each state. The transitions are:
*
* RUNNING -> SHUTDOWN
* On invocation of shutdown(), perhaps implicitly in finalize()
* (RUNNING or SHUTDOWN) -> STOP
* On invocation of shutdownNow()
* SHUTDOWN -> TIDYING
* When both queue and pool are empty
* STOP -> TIDYING
* When pool is empty
* TIDYING -> TERMINATED
* When the terminated() hook method has completed
*
* Threads waiting in awaitTermination() will return when the
* state reaches TERMINATED.
*
* Detecting the transition from SHUTDOWN to TIDYING is less
* straightforward than you'd like because the queue may become
* empty after non-empty and vice versa during SHUTDOWN state, but
* we can only terminate if, after seeing that it is empty, we see
* that workerCount is 0 (which sometimes entails a recheck -- see
* below).
*/
private final AtomicInteger ctl = new AtomicInteger(ctlOf(RUNNING, 0));
private static final int COUNT_BITS = Integer.SIZE - 3;
private static final int CAPACITY = (1 << COUNT_BITS) - 1;
// runState is stored in the high-order bits
private static final int RUNNING = -1 << COUNT_BITS;
private static final int SHUTDOWN = 0 << COUNT_BITS;
private static final int STOP = 1 << COUNT_BITS;
private static final int TIDYING = 2 << COUNT_BITS;
private static final int TERMINATED = 3 << COUNT_BITS;
// Packing and unpacking ctl
private static int runStateOf(int c) { return c & ~CAPACITY; }
private static int workerCountOf(int c) { return c & CAPACITY; }
private static int ctlOf(int rs, int wc) { return rs | wc; }
private static boolean isRunning(int c) {
return c < SHUTDOWN;
}
相关源码解读:
不知大家有没有想过为什么线程池的状态简单的定义为 -1,0,1,2,3 不就得了,为什么还要用移位操作呢?
原来这样的,ThreadPool ctl 的这个变量的设计哲学是用int的高3位 + 29个0代表状态,用高位000+低29位来表示线程池中工作线程的数量,太佩服了。
首先 CAPACITY 的值为 workCount 的最大容量,该值为 000 11111 11111111 11111111 11111111,29个1,
我们来看一下
private static int runStateOf(int c) { return c & ~CAPACITY; }
用ctl里面的值与容量取反的方式获取状态值。由于CAPACITY的值为000 11111 11111111 11111111 11111111,那取反后为111 00000 00000000 00000000 00000000, 用 c 与 该值进行与运算,这样就直接保留了c的高三位,然后将c的低29位设置为0,这不就是线程池状态的存放规则吗,绝。
根据此方法,不难得出计算workCount的方法。
private static int ctlOf(int rs, int wc) { return rs | wc; }
该方法,主要是用来更新运行状态的。确保工作线程数量不丢失。
线程池状态以及含义。
- RUNNING 运行态
- SHUTDOWN 关闭,此时不接受新的任务,但继续处理队列中的任务。
- STOP 停止,此时不接受新的任务,不处理队列中的任务,并中断正在执行的任务
- TIDYING 所有的工作线程全部停止,并工作线程数量为0,将调用terminated方法,进入到TERMINATED
- TERMINATED 终止状态
线程池默认状态 RUNNING
如果调用shutdwon() 方法,状态从 RUNNING ---> SHUTDOWN。
如果调用shutdwonNow()方法,状态从RUUNING|SHUTDOWN--->STOP。
SHUTDOWN ---> TIDYING 。
队列为空并且线程池空。
STOP --> TIDYING。
线程池为空。
线程池设计原理:
- 线程池的工作线程为ThreadPoolExecutors的Worker线程,无论是submit还是executor方法中传入的Callable task,Runable参数,只是实现了Runnable接口,在线程池的调用过程,不会调用其start方法,只会调用Worker线程的start方法,然后在Worker线程的run方法中会调用入参的run方法。
- 众所周知,线程的生命周期在run方法运行结束后(包括异常退出)就结束。要想重复利用线程,我们要确保工作线程Worker的run方法运行在一个无限循环中,然后从任务队列中一个一个获取对象,如果任务队列为空,则阻塞,当然需要提供一些控制,结束无限循环,来销毁线程。在源码 runWorker方法与getTask来实现。
大概的实现思路是 如果 getTask 返回null, 则该 worker 线程将被销毁。
那 getTask 在什么情况下会返回 false 呢?
- 如果线程池的状态为SHUTDOWN并且队列不为空
- 如果线程池的状态大于STOP
- 如果当前运行的线程数大于核心线程数,会返回null,已销毁该worker线程
对keepAliveTime的理解,如果allowCoreThreadTimeOut为真,那么keepAliveTime其实就是从任务队列获取任务等待的超时时间,也就是workerQueue.poll(keepALiveTime, TimeUnit.NANOSECONDS)
3.2 <T> FUture<T> submit(Callable<T> task) 方法详解
在看的代码的过程中,只要明白了上述基础方法,后面的代码看起来清晰可见,故,我只列出关键方法,大家可以浏览,应该不难。
/**
* Submits a value-returning task for execution and returns a
* Future representing the pending results of the task. The
* Future's <tt>get</tt> method will return the task's result upon
* successful completion.
*
* <p>
* If you would like to immediately block waiting
* for a task, you can use constructions of the form
* <tt>result = exec.submit(aCallable).get();</tt>
*
* <p> Note: The {@link Executors} class includes a set of methods
* that can convert some other common closure-like objects,
* for example, {@link java.security.PrivilegedAction} to
* {@link Callable} form so they can be submitted.
*
* @param task the task to submit
* @return a Future representing pending completion of the task
* @throws RejectedExecutionException if the task cannot be
* scheduled for execution
* @throws NullPointerException if the task is null
*/
<T> Future<T> submit(Callable<T> task);
提交一个任务,并返回结构到Future,Future就是典型的Future设计模式,就是提交任务到线程池后,返回一个凭证,并直接返回,主线程继续执行,然后当线程池将任务运行完毕后,再将结果填充到凭证中,当主线程调用凭证future的get方法时,如果结果还未填充,则阻塞等待。
现将Callable与Future接口的源代码贴出来,然后重点分析submit方法的实现。
public interface Callable<V> {
/**
* Computes a result, or throws an exception if unable to do so.
*
* @return computed result
* @throws Exception if unable to compute a result
*/
V call() throws Exception;
}
public interface Future<V> {
/**
* Attempts to cancel execution of this task. This attempt will
* fail if the task has already completed, has already been cancelled,
* or could not be cancelled for some other reason. If successful,
* and this task has not started when <tt>cancel</tt> is called,
* this task should never run. If the task has already started,
* then the <tt>mayInterruptIfRunning</tt> parameter determines
* whether the thread executing this task should be interrupted in
* an attempt to stop the task.
*
* <p>After this method returns, subsequent calls to {@link #isDone} will
* always return <tt>true</tt>. Subsequent calls to {@link #isCancelled}
* will always return <tt>true</tt> if this method returned <tt>true</tt>.
*
* @param mayInterruptIfRunning <tt>true</tt> if the thread executing this
* task should be interrupted; otherwise, in-progress tasks are allowed
* to complete
* @return <tt>false</tt> if the task could not be cancelled,
* typically because it has already completed normally;
* <tt>true</tt> otherwise
*/
boolean cancel(boolean mayInterruptIfRunning);
/**
* Returns <tt>true</tt> if this task was cancelled before it completed
* normally.
*
* @return <tt>true</tt> if this task was cancelled before it completed
*/
boolean isCancelled();
/**
* Returns <tt>true</tt> if this task completed.
*
* Completion may be due to normal termination, an exception, or
* cancellation -- in all of these cases, this method will return
* <tt>true</tt>.
*
* @return <tt>true</tt> if this task completed
*/
boolean isDone();
/**
* Waits if necessary for the computation to complete, and then
* retrieves its result.
*
* @return the computed result
* @throws CancellationException if the computation was cancelled
* @throws ExecutionException if the computation threw an
* exception
* @throws InterruptedException if the current thread was interrupted
* while waiting
*/
V get() throws InterruptedException, ExecutionException;
/**
* Waits if necessary for at most the given time for the computation
* to complete, and then retrieves its result, if available.
*
* @param timeout the maximum time to wait
* @param unit the time unit of the timeout argument
* @return the computed result
* @throws CancellationException if the computation was cancelled
* @throws ExecutionException if the computation threw an
* exception
* @throws InterruptedException if the current thread was interrupted
* while waiting
* @throws TimeoutException if the wait timed out
*/
V get(long timeout, TimeUnit unit)
throws InterruptedException, ExecutionException, TimeoutException;
}
现在开始探究submit的实现原理,该代码出自AbstractExecutorService中
public Future<?> submit(Runnable task) {
if (task == null) throw new NullPointerException();
RunnableFuture<Void> ftask = newTaskFor(task, null);
execute(ftask);
return ftask;
}
protected <T> RunnableFuture<T> newTaskFor(Callable<T> callable) {
return new FutureTask<T>(callable);
}
核心实现在ThreadPoolExecutor的execute方法
/**
* Executes the given task sometime in the future. The task
* may execute in a new thread or in an existing pooled thread.
*
* If the task cannot be submitted for execution, either because this
* executor has been shutdown or because its capacity has been reached,
* the task is handled by the current {@code RejectedExecutionHandler}.
*
* @param command the task to execute
* @throws RejectedExecutionException at discretion of
* {@code RejectedExecutionHandler}, if the task
* cannot be accepted for execution
* @throws NullPointerException if {@code command} is null
*/
public void execute(Runnable command) {
if (command == null)
throw new NullPointerException();
/*
* Proceed in 3 steps:
*
* 1. If fewer than corePoolSize threads are running, try to
* start a new thread with the given command as its first
* task. The call to addWorker atomically checks runState and
* workerCount, and so prevents false alarms that would add
* threads when it shouldn't, by returning false.
*
* 2. If a task can be successfully queued, then we still need
* to double-check whether we should have added a thread
* (because existing ones died since last checking) or that
* the pool shut down since entry into this method. So we
* recheck state and if necessary roll back the enqueuing if
* stopped, or start a new thread if there are none.
*
* 3. If we cannot queue task, then we try to add a new
* thread. If it fails, we know we are shut down or saturated
* and so reject the task.
*/
int c = ctl.get();
if (workerCountOf(c) < corePoolSize) { // @1
if (addWorker(command, true)) // @2
return;
c = ctl.get(); //@3
}
if (isRunning(c) && workQueue.offer(command)) {
int recheck = ctl.get();
if (! isRunning(recheck) && remove(command))
reject(command);
else if (workerCountOf(recheck) == 0)
addWorker(null, false);
}
else if (!addWorker(command, false))
reject(command);
}
代码@1,如果当前线程池中的线程数量小于核心线程数的话,尝试新增一个新的线程。所以我们把目光投入到addWorker方法中。
addWorker源码详解:
/**
* Checks if a new worker can be added with respect to current
* pool state and the given bound (either core or maximum). If so,
* the worker count is adjusted accordingly, and, if possible, a
* new worker is created and started, running firstTask as its
* first task. This method returns false if the pool is stopped or
* eligible to shut down. It also returns false if the thread
* factory fails to create a thread when asked. If the thread
* creation fails, either due to the thread factory returning
* null, or due to an exception (typically OutOfMemoryError in
* Thread#start), we roll back cleanly.
*
* @param firstTask the task the new thread should run first (or
* null if none). Workers are created with an initial first task
* (in method execute()) to bypass queuing when there are fewer
* than corePoolSize threads (in which case we always start one),
* or when the queue is full (in which case we must bypass queue).
* Initially idle threads are usually created via
* prestartCoreThread or to replace other dying workers.
*
* @param core if true use corePoolSize as bound, else
* maximumPoolSize. (A boolean indicator is used here rather than a
* value to ensure reads of fresh values after checking other pool
* state).
* @return true if successful
*/
private boolean addWorker(Runnable firstTask, boolean core) {
retry:
for (;;) { // @1
int c = ctl.get();
int rs = runStateOf(c); // @2
// Check if queue empty only if necessary.
if (rs >= SHUTDOWN && //@3
! (rs == SHUTDOWN &&
firstTask == null &&
! workQueue.isEmpty()))
return false;
for (;;) { //@4
int wc = workerCountOf(c);
if (wc >= CAPACITY ||
wc >= (core ? corePoolSize : maximumPoolSize)) //@5
return false;
if (compareAndIncrementWorkerCount(c))
break retry; // @6
c = ctl.get(); // Re-read ctl
if (runStateOf(c) != rs)
continue retry; //@7
// else CAS failed due to workerCount change; retry inner loop
}
}
boolean workerStarted = false;
boolean workerAdded = false;
Worker w = null;
try {
final ReentrantLock mainLock = this.mainLock;
w = new Worker(firstTask);
final Thread t = w.thread;
if (t != null) {
mainLock.lock(); // @8
try {
// Recheck while holding lock.
// Back out on ThreadFactory failure or if
// shut down before lock acquired.
int c = ctl.get();
int rs = runStateOf(c);
if (rs < SHUTDOWN ||
(rs == SHUTDOWN && firstTask == null)) {
if (t.isAlive()) // precheck that t is startable
throw new IllegalThreadStateException();
workers.add(w);
int s = workers.size();
if (s > largestPoolSize)
largestPoolSize = s;
workerAdded = true;
}
} finally {
mainLock.unlock();
}
if (workerAdded) {
t.start(); // 运行线程 // @9
workerStarted = true;
} //@8 end
}
} finally {
if (! workerStarted)
addWorkerFailed(w); // 增加工作线程失败
}
return workerStarted;
}
代码@1,外层循环(自旋模式)
代码@2,获取线程池的运行状态
代码@3,这里的判断条件,为什么不直接写 if(rs >= SHUTDOWN) return false;而要加第二个条件,目前不明白,等在了解到参数firstTask在什么情况下为空。在这里,我们目前只要知道,只有线程池的状态为 RUNNING时,线程池才接收新的任务,去增加工作线程。
代码@4,内层循环,主要的目的就是利用CAS增加一个线程数量。
代码@5,判断当前线程池的数量,如果数量达到规定的数量,则直接返回false,添加工作线程失败。
代码@6,如果修改线程数量成功,则跳出循环,开始创建工作线程。
代码@7,如果修改线程数量不成功(CAS)有两种情况:1、线程数量变化,重试则好,2,如果线程的运行状态变化,则重新开始外层循环,重新判断addWork流程。
代码@8,在锁mainLock的保护下,完成 workers (HashSet)的维护。
接着分析一下代码@9,启动线程,执行关键的方法 runWorker方法:
/**
* Main worker run loop. Repeatedly gets tasks from queue and
* executes them, while coping with a number of issues:
*
* 1. We may start out with an initial task, in which case we
* don't need to get the first one. Otherwise, as long as pool is
* running, we get tasks from getTask. If it returns null then the
* worker exits due to changed pool state or configuration
* parameters. Other exits result from exception throws in
* external code, in which case completedAbruptly holds, which
* usually leads processWorkerExit to replace this thread.
*
* 2. Before running any task, the lock is acquired to prevent
* other pool interrupts while the task is executing, and
* clearInterruptsForTaskRun called to ensure that unless pool is
* stopping, this thread does not have its interrupt set.
*
* 3. Each task run is preceded by a call to beforeExecute, which
* might throw an exception, in which case we cause thread to die
* (breaking loop with completedAbruptly true) without processing
* the task.
*
* 4. Assuming beforeExecute completes normally, we run the task,
* gathering any of its thrown exceptions to send to
* afterExecute. We separately handle RuntimeException, Error
* (both of which the specs guarantee that we trap) and arbitrary
* Throwables. Because we cannot rethrow Throwables within
* Runnable.run, we wrap them within Errors on the way out (to the
* thread's UncaughtExceptionHandler). Any thrown exception also
* conservatively causes thread to die.
*
* 5. After task.run completes, we call afterExecute, which may
* also throw an exception, which will also cause thread to
* die. According to JLS Sec 14.20, this exception is the one that
* will be in effect even if task.run throws.
*
* The net effect of the exception mechanics is that afterExecute
* and the thread's UncaughtExceptionHandler have as accurate
* information as we can provide about any problems encountered by
* user code.
*
* @param w the worker
*/
final void runWorker(Worker w) {
Thread wt = Thread.currentThread();
Runnable task = w.firstTask;
w.firstTask = null;
w.unlock(); // allow interrupts
boolean completedAbruptly = true;
try {
while (task != null || (task = getTask()) != null) {
w.lock();
// If pool is stopping, ensure thread is interrupted;
// if not, ensure thread is not interrupted. This
// requires a recheck in second case to deal with
// shutdownNow race while clearing interrupt
if ((runStateAtLeast(ctl.get(), STOP) ||
(Thread.interrupted() &&
runStateAtLeast(ctl.get(), STOP))) &&
!wt.isInterrupted())
wt.interrupt();
try {
beforeExecute(wt, task);
Throwable thrown = null;
try {
task.run();
} catch (RuntimeException x) {
thrown = x; throw x;
} catch (Error x) {
thrown = x; throw x;
} catch (Throwable x) {
thrown = x; throw new Error(x);
} finally {
afterExecute(task, thrown);
}
} finally {
task = null;
w.completedTasks++;
w.unlock();
}
}
completedAbruptly = false;
} finally {
processWorkerExit(w, completedAbruptly);
}
}
/**
* Performs blocking or timed wait for a task, depending on
* current configuration settings, or returns null if this worker
* must exit because of any of:
* 1. There are more than maximumPoolSize workers (due to
* a call to setMaximumPoolSize).
* 2. The pool is stopped.
* 3. The pool is shutdown and the queue is empty.
* 4. This worker timed out waiting for a task, and timed-out
* workers are subject to termination (that is,
* {@code allowCoreThreadTimeOut || workerCount > corePoolSize})
* both before and after the timed wait.
*
* @return task, or null if the worker must exit, in which case
* workerCount is decremented
*/
private Runnable getTask() {
boolean timedOut = false; // Did the last poll() time out?
retry:
for (;;) {
int c = ctl.get();
int rs = runStateOf(c);
// Check if queue empty only if necessary.
if (rs >= SHUTDOWN && (rs >= STOP || workQueue.isEmpty())) {
decrementWorkerCount();
return null;
}
boolean timed; // Are workers subject to culling?
for (;;) {
int wc = workerCountOf(c);
timed = allowCoreThreadTimeOut || wc > corePoolSize;
if (wc <= maximumPoolSize && ! (timedOut && timed))
break;
if (compareAndDecrementWorkerCount(c))
return null;
c = ctl.get(); // Re-read ctl
if (runStateOf(c) != rs)
continue retry;
// else CAS failed due to workerCount change; retry inner loop
}
try {
Runnable r = timed ?
workQueue.poll(keepAliveTime, TimeUnit.NANOSECONDS) :
workQueue.take();
if (r != null)
return r;
timedOut = true;
} catch (InterruptedException retry) {
timedOut = false;
}
}
}
欢迎加笔者微信号(dingwpmz),加群探讨,笔者优质专栏目录:
1、源码分析RocketMQ专栏(40篇+)
2、源码分析Sentinel专栏(12篇+)
3、源码分析Dubbo专栏(28篇+)
4、源码分析Mybatis专栏
5、源码分析Netty专栏(18篇+)
6、源码分析JUC专栏
7、源码分析Elasticjob专栏
8、Elasticsearch专栏
9、源码分析Mycat专栏