主要的并发访问变量:
1. ctl(3位:状态位 + 29位:workerCount)
2. workers工作线程的集合以及一些统计数据,使用mainLock控制保护
3. Worker的中断标志,使用AbstractQueuedSynchronizer中的state形成lock和unlock保护
管理线程池线程：
execute addWorker addWorkerFailed tryTerminate interruptIdleWorkers interruptWorkers shutdown shutdownNow interruptIfStarted
线程池中的工作线程:
runWorker processWorkerExit getTask
线程池状态(状态只能逐渐变大),只有管理线程会修改状态:
RUNNING：可以接受新的任务，也可以处理阻塞队列里的任务
SHUTDOWN：不接受新的任务，但是可以处理阻塞队列里的任务
STOP：不接受新的任务，不处理阻塞队列里的任务，中断正在处理的任务
TIDYING：过渡状态，也就是说所有的任务都执行完了，当前线程池已经没有有效的线程，这个时候线程池的状态将会TIDYING，并且将要调用terminated方法
TERMINATED：终止状态。terminated方法调用完成以后的状态
状态变化迁移:
RUNNING -> SHUTDOWN：手动调用shutdown方法，或者ThreadPoolExecutor要被GC回收的时候调用finalize方法，finalize方法内部也会调用shutdown方法
(RUNNING or SHUTDOWN) -> STOP：调用shutdownNow方法，可以在之前调用过shutdown
SHUTDOWN -> TIDYING：当队列和线程池都为空的时候
STOP -> TIDYING：当线程池为空的时候
TIDYING -> TERMINATED：terminated方法调用完成之后
ThreadPoolExecutor内部还保存着线程池的有效线程个数。
线程池数量:
一般来说管理线程会增加workerCount
工作线程退出时减少workerCount
但是在管理线程addWorker中没有成功启动一个线程是会回退workerCount
关闭方法总结：
shutdown方法会更新状态到SHUTDOWN，不会影响阻塞队列里任务的执行，但是不会执行新进来的任务。同时也会回收闲置的Worker，闲置Worker(没有获得worker中lock)。
shutdownNow方法会更新状态到STOP，会影响阻塞队列的任务执行，也不会执行新进来的任务。同时会回收所有的Worker。
/**
* 继承AbstractQueuedSynchronizer为了实现对中断标志的管理以及防止在runWorker前就被interrupt
* 在每次执行task前会lock，在中断前会tryLock
*/
Worker线程
1. runWorker
* 1. unlock 设置worker.status = 0，锁初始化状态（未锁）
* 2.for (;;)
* 1. getTask获得任务 或者 初始化task不为null
* 2. worker.lock其他线程不允许设置worker的中断标志
* 3. 1. 如果线程池已经处于>=STOP状态并且当前线程没有被中断，中断线程
2. 如果线程池还处于RUNNING或SHUTDOWN状态，并且当前线程已经被中断了，重新检查一下线程池状态，(跟shutdownNow有竞争）
3. 如果处于STOP状态并且没有被中断
那么设置中断线程worker.interrupt，表示可以退出
* 4. task.run运行任务 (如果task执行发现异常，completedAbruptly=true)
5. finally worker.unlock
* 3. processWorkerExit
2. processWorkerExit
* 1. 异常退出，补偿一次 decrementWorkerCount 修改 workerCount
* 2. mainLock.lock
* 3. 修改workers删除worker
* 4. mainLock.unlock
* 5. tryTerminate
* 6. 1. 用户执行的任务发生了异常
2. Worker数量比线程池基本大小要小
3. 阻塞队列不空但是没有任何Worker在工作
addWorker(null, false); 添加一个线程
3. getTask
* 1. 大于等于STOP 或 为SHUTDOWN && 队列已经为空，decrementWorkerCount，返回null
* 2. 超时 && 线程数超过(corePoolSize || 设置了allowCoreThreadTimeOut) && (wc > 1 || 队列为空)
decrementWorkerCount，返回null，用于回收多余的worker
* 3. take或者poll从队列中获取task
管理线程
1. execute
* 1.小于corePoolSize直接addWorker
* 2.>= corePoolSize&&正在运行中, 放入workQueue中排队处理
* offer后再次检查，因为在之前ctl.get后到offer完成前有时间窗口，比如：offer完成前线程池状态从RUNNING->TERMINATED这时，放到队列
中的任务永远被遗忘了
1.如果为非running状态，remove
2.检查是否有worker，因为command已经在queue中了需要有worker处理
* 3.线程池状态不是RUNNING或者workercount>=corePoolSize,
* addWorker(command, false)，在addWorker函数会过滤线程池状态，考虑正在shutdown的情况
2. addWorker
* 1. 1.线程池不在RUNNING状态并且状态是STOP、TIDYING或TERMINATED中的任意一种状态
* 2.线程池不在RUNNING状态，线程池接受了新的任务firstTask!=null
* 3. 线程池不在RUNNING状态，阻塞队列为空
这些情况不增加worker
* 2. compareAndIncrementWorkerCount 更新 workerCount
* 3. 创建worker
* 4. mainLock.lock后更新workers集合
* 5. 启动新创建的worker---------------------worker正式开始工作但是状态还是-1，跑到runWorker后unlock才会变成0
* 6. 如果启动是否addWorkerFailed
3. addWorkerFailed 创建worker时失败（worker没有正常启动）
* 1. mainLock.lock
* 2. decrementWorkerCount
* 3. tryTerminate
4. tryTerminate(在worker减少或者removing tasks时被调用)
* 1. 如果线程状态为 Running、TERMINATED,TIDYING（已经在结束的路上了）、SHUTDOWN但是workQueue不为空不允许退出
* 2. 到这里说明状态为shutdown&&队列为空
如果workerCount != 0， interruptIdleWorkers 中断一个空闲的worker，
在worker回收最后processWorkerExit还会调用tryTerminate
* 3. compareAndSet 设置 status-TIDYING
* 4. terminated
* 5. 设置 compareAndSet status-TERMINATED最终结束
5. interruptIdleWorkers
* 1. mainLock.lock
* 2. 遍历workers，如果线程没有中断 && worker.tryLock（如果正确运行task不会成功）成功
设置worker的中断标志
* 3. 如果只打断一个，成功一个后退出
6. interruptWorkers
* 1. mainLock.lock
* 2. 遍历所有workers，interruptIfStarted(刚初始化没有运行到runWorker的会被过滤)
7. shutdown
* 1. advanceRunState SHUTDOWN: 原来状态大于等于targetState 或者 compareAndSet 设置 status 为 SHUTDOWN or STOP
* 2. interruptIdleWorkers(false)设置所有worker的中断标志
* 3. tryTerminate
8. shutdownNow
* 1. advanceRunState STOP: 原来状态大于等于targetState 或者 compareAndSet 设置 status 为 SHUTDOWN or STOP
* 2. interruptWorkers
* 3. drainQueue将没有跑完的任务返回
9. interruptIfStarted(worker中方法)
* 1. 如果状态不是为-1(还没有进入runWorker，只是初始化状态) && 没有中断标志
设置中断标志
假设管理线程调用shutdownNow-STOP，但是某个worker在调用interruptIfStarted时还是初始化状态，
随后worker会进入runWorker函数发现线程池状态以及是STOP状态，所以会直接退出
主要函数注释：
public class ThreadPoolExecutor extends AbstractExecutorService {
private final AtomicInteger ctl = new AtomicInteger(ctlOf(RUNNING, 0));
private static final int COUNT_BITS = Integer.SIZE - 3; //29
private static final int CAPACITY = (1 << COUNT_BITS) - 1; //1fffffff
// runState is stored in the high-order bits
private static final int RUNNING = -1 << COUNT_BITS; //-536870912 e0000000, 初始化状态
private static final int SHUTDOWN = 0 << COUNT_BITS; //0 0
private static final int STOP = 1 << COUNT_BITS; //536870912 20000000
private static final int TIDYING = 2 << COUNT_BITS; //1073741824 40000000
private static final int TERMINATED = 3 << COUNT_BITS; //1610612736 60000000
// 获得state
private static int runStateOf(int c) { return c & ~CAPACITY; }
// 获得count
private static int workerCountOf(int c) { return c & CAPACITY; }
// 使用state和count拼接处ctl值
private static int ctlOf(int rs, int wc) { return rs | wc; }
private static boolean runStateLessThan(int c, int s) {
return c < s;
}
private static boolean runStateAtLeast(int c, int s) {
return c >= s;
}
private static boolean isRunning(int c) {
return c < SHUTDOWN;
}
/**
* Attempts to CAS-increment the workerCount field of ctl.
*/
private boolean compareAndIncrementWorkerCount(int expect) {
return ctl.compareAndSet(expect, expect + 1);
}
/**
* Attempts to CAS-decrement the workerCount field of ctl.
*/
private boolean compareAndDecrementWorkerCount(int expect) {
return ctl.compareAndSet(expect, expect - 1);
}
/**
* Decrements the workerCount field of ctl. This is called only on
* abrupt termination of a thread (see processWorkerExit). Other
* decrements are performed within getTask.
*/
private void decrementWorkerCount() {
do {} while (! compareAndDecrementWorkerCount(ctl.get()));
}
private BlockingDeque<Runnable> workQueue;
private volatile RejectedExecutionHandler handler;
private static final RejectedExecutionHandler defaultHandler = new AbortPolicy();
private final void reject(Runnable command) {
handler.rejectedExecution(command, this);
}
/**
* 用于workers集合以及相关统计信息并发控制
* shutdown shutdownNow interruptIdleWorkers interruptWorkers
* processWorkerExit addWorker termination 以及 getXXX都会用到
*/
private final ReentrantLock mainLock = new ReentrantLock();
private final HashSet<Worker> workers = new HashSet<>();
/**
* tryTerminate-signal awaitTermination-await中使用
*/
private final Condition termination = mainLock.newCondition();
/**
* 在mainLock下使用
*/
private long completedTaskCount;
private volatile int corePoolSize;
private volatile int maximumPoolSize;
// 到目前为止出现过最大的数量
private int largestPoolSize;
private volatile ThreadFactory threadFactory;
//一般使用在idle线程，如果allowCoreThreadTimeOut设置为true也适用
private volatile long keepAliveTime;
//默认为false，如果为true表示core threads使用keepAliveTime超时时间
private volatile boolean allowCoreThreadTimeOut;
protected void beforeExecute(Thread t, Runnable r) { }
protected void afterExecute(Runnable r, Throwable t) { }
public BlockingQueue<Runnable> getQueue() {
return workQueue;
}
private void interruptIdleWorkers(boolean onlyOne) {
final ReentrantLock mainLock = this.mainLock;
mainLock.lock();
try {
for (Worker w : workers) {
Thread t = w.thread;
if (!t.isInterrupted() && w.tryLock()) {
try {
t.interrupt();
} catch (SecurityException ignore) {
} finally {
w.unlock();
}
}
if (onlyOne)
break;
}
} finally {
mainLock.unlock();
}
}
protected void terminated() { }
/*
*This method must be called following any action that might make
* termination possible -- reducing worker count or removing tasks
* from the queue during shutdown.
*/
private void tryTerminate() {
for (;;) {
int c = ctl.get();
/**
* 1.running
* 2.TERMINATED,TIDYING
* 3.SHUTDOWN但是workQueue不为空
*/
if (isRunning(c) || runStateAtLeast(c, TIDYING) || (runStateOf(c) == SHUTDOWN && ! workQueue.isEmpty())) {
return;
}
// 到这里说明不为running状态，如果shutdown状态队列也为空了，可以回收workers
if (workerCountOf(c) != 0) { // Eligible to terminate
// 中断闲置一个Worker，直到回收全部的Worker。
// 这里没有那么暴力，只中断一个，中断之后退出方法，中断了Worker之后，Worker会回收，
// 在processWorkerExit中然后还是会调用tryTerminate方法，如果还有闲置线程，那么继续中断
interruptIdleWorkers(true);
return;
}
// 这里到这里说明不为running状态，如果shutdown状态队列也为空了，工作线程为0
final ReentrantLock mainLock = this.mainLock;
mainLock.lock();
try {
// cas操作，将线程池状态改成TIDYING
if (ctl.compareAndSet(c, ctlOf(TIDYING, 0))){
try {
// 从TIDYING ->TERMINATED给一次回调的机会
terminated();
} finally {
// terminated方法调用完毕之后，状态变为TERMINATED
ctl.set(ctlOf(TERMINATED, 0));
// 通知awaitTerminate的线程
termination.signalAll();
}
return;
}
} finally {
mainLock.unlock(); // 解锁
}
}
}
private void processWorkerExit(Worker w, boolean completedAbruptly) {
// 异常退出不会再getTask中decrementWorkerCount
if (completedAbruptly) {
decrementWorkerCount();
}
final ReentrantLock mainLock = this.mainLock;
mainLock.lock();
try {
completedTaskCount += w.completedTasks;
workers.remove(w);
} finally {
mainLock.unlock();
}
// 尝试结束线程池，满足条件会调用interruptIdleWorkers
tryTerminate();
int c = ctl.get();
if (runStateLessThan(c,STOP)) {
if (!completedAbruptly) {
//正常结束情况
int min = allowCoreThreadTimeOut ? 0 : corePoolSize;
if (min == 0 && !workQueue.isEmpty())
min = 1;
// 不需要开线程
if (workerCountOf(c) >= min)
return;
}
// 新开一个Worker代替原先的Worker
// 新开一个Worker需要满足以下3个条件中的任意一个：
// 1. 用户执行的任务发生了异常
// 2. Worker数量比线程池基本大小要小
// 3. 阻塞队列不空但是没有任何Worker在工作
addWorker(null, false);
}
}
private void addWorkerFailed(Worker w) {
final ReentrantLock mainLock = this.mainLock;
mainLock.lock();
try {
if (w != null)
workers.remove(w);
decrementWorkerCount();
tryTerminate();
} finally {
mainLock.unlock();
}
}
/**
*
* @param firstTask 如果为null，表示这个worker起来是为了从队列中获得task
* 否则表示问因为有新任务execute引起worker的增加
* @param core 是否检查当前worker >= corePoolSize
* @return true：成功添加并且worker已经started
* false：失败
*/
private boolean addWorker(Runnable firstTask, boolean core) {
retry:
for (; ; ) { // #1
int c = ctl.get();
int rs = runStateOf(c);
/**
* 1. 线程池不在RUNNING状态并且状态是STOP、TIDYING或TERMINATED中的任意一种状态
* 2. 线程池不在RUNNING状态，线程池接受了新的任务firstTask!=null
* 3. 线程池不在RUNNING状态，阻塞队列为空
*/
if (rs > SHUTDOWN || (rs == SHUTDOWN && firstTask != null) || (rs == SHUTDOWN && workQueue.isEmpty()))
return false;
for (; ; ) { // #3
int wc = workerCountOf(c);
/**
* 超过线程数
*/
if (wc >= CAPACITY || wc >= (core ? corePoolSize : maximumPoolSize))
return false;
// 原子修改workercount，成功跳出两层循环
if (compareAndIncrementWorkerCount(c))
break retry; // 跳转到 #2
c = ctl.get();
// 检查rs时发现有其他线程改变了，重新取ctl
if (runStateOf(c) != rs)
continue retry; // 跳转到 #1
else {
// 如果状态没有被人改变跳转到 #3
}
}
}
// #2
boolean workerStarted = false;
boolean workerAdded = false;
Worker w = null;
try {
w = new Worker(firstTask);
final Thread t = w.thread;
if (t != null) {
final ReentrantLock mainLock = this.mainLock;
mainLock.lock();
try {
int rs = runStateOf(ctl.get());
/**
* 1. running 状态
* 2. 如果为shutdown状态并且task为null
*/
if (rs < SHUTDOWN || (rs == SHUTDOWN && firstTask ==null)) {
//已经启动过了
if (t.isAlive())
throw new IllegalThreadStateException();
workers.add(w);
int s = workers.size();
if (s > largestPoolSize)
largestPoolSize = s;
workerAdded = true;
}
} finally {
mainLock.unlock();
}
if (workerAdded) {
t.start();
workerStarted = true;
}
}
} finally {
if (!workerStarted) {
addWorkerFailed(w);
}
}
return workerStarted;
}
public boolean remove(Runnable task) {
boolean removed = workQueue.remove(task);
tryTerminate(); // In case SHUTDOWN and now empty
return removed;
}
public void execute(Runnable command) {
if (command == null)
throw new NullPointerException();
/**
* 1.小于corePoolSize直接addWorker
* 2.>= corePoolSize&&正在运行中, 放入workQueue中排队处理
* offer后再次检查1.如果为非running状态，remove 2.检查是否有worker，因为command已经在queue中了需要有worker处理
* 3.线程池状态不是RUNNING或者workercount>=corePoolSize,
* addWorker(command, false)，在addWorker函数会过滤线程池状态，考虑正在shutdown的情况
*/
int c = ctl.get();
if (workerCountOf(c) < corePoolSize) {
if (addWorker(command, true))
return;
// 这里再取为了缩短时间窗口
c = ctl.get();
}
if (isRunning(c) && workQueue.offer(command)) {
int recheck = ctl.get();
//放入后再做一次检查，因为在ctl get到offer完成前有时间窗口，其他线程会修改ctl，所以需要recheck
//如果offer完毕尝试remove，如果可以remove就reject
//否则判断当前是否还有worker活着，如果没有活着的worker，添加一个worker让这个worker处理task
if (!isRunning(recheck) && remove(command))
reject(command);
else if (workerCountOf(recheck) == 0)
addWorker(null, false);
}
//线程池状态不是RUNNING或者workercount>=corePoolSize
else if (!addWorker(command, false))
reject(command);
}
/**
* 1. 状态为>=STOP或者(rs==SHUTDOWN并且workQueue空)
* 2. 超过maximumPoolSize
* @return task
* null: 返回null前会compareAndDecrementWorkerCount表示当前worker需退出
*/
private Runnable getTask() {
boolean timedOut = false;
for (;;) {
int c = ctl.get();
int rs = runStateOf(c);
// 大于等于STOP 或 为SHUTDOWN && 队列已经为空
// if (rs >= STOP || (rs == SHUTDOWN && workQueue.isEmpty())
if (rs >= SHUTDOWN && (rs >= STOP || workQueue.isEmpty())) {
decrementWorkerCount();
return null;
}
int wc = workerCountOf(c);
boolean timed = allowCoreThreadTimeOut || wc > corePoolSize;
if ((wc > maximumPoolSize || (timed && timedOut)) &&
(wc > 1 || workQueue.isEmpty())) {
if (compareAndDecrementWorkerCount(c))
return null;
continue;
}
try {
Runnable r = timed ? workQueue.poll(keepAliveTime, TimeUnit.MICROSECONDS)
: workQueue.take();
if (r != null)
return r;
timedOut = true;
} catch (InterruptedException e) {
timedOut = false;
}
}
}
private void runWorker(Worker worker) {
Thread wt = Thread.currentThread();
Runnable task = worker.firstTask;
worker.firstTask = null;
worker.unlock(); // 设置state=0，允许interrupts
boolean completedAbruptly = true;
try {
// getTask阻塞的等待task或者超时
while (task != null || (task = getTask()) != null) {
// 如果拿到了任务，给自己上锁，表示当前Worker已经要开始执行任务了，
// 已经不是闲置Worker,有lock表示非闲置
worker.lock();
// 1. 如果线程池已经处于>=STOP状态并且当前线程没有被中断，中断线程
// 2. 如果线程池还处于RUNNING或SHUTDOWN状态，并且当前线程已经被中断了，重新检查一下线程池状态，(跟shutdownNow有竞争）
// 如果处于STOP状态并且没有被中断，那么中断线程
if ((runStateAtLeast(ctl.get(), STOP) || (Thread.interrupted() && runStateAtLeast(ctl.get(), STOP)))
&& !wt.isInterrupted()) {
//设置为线程中断
wt.interrupt();
}
try {
beforeExecute(wt, task);
Throwable thrown = null;
try {
//真正执行任务
task.run();
} catch (RuntimeException x) {
thrown = x;
throw x;
} catch (Error x) {
thrown = x;
throw x;
} catch (Throwable x) {
thrown = x;
throw new Error(x);
} finally {
afterExecute(wt, thrown);
}
} finally {
task = null;
worker.completedTasks++;
worker.unlock();
}
}
//正常结束
completedAbruptly = false;
} finally {
// worker退出
processWorkerExit(worker, completedAbruptly);
}
}
public ThreadFactory getThreadFactory() {
return threadFactory;
}
/**
* 继承AbstractQueuedSynchronizer为了实现对中断标志的管理以及防止在runWorker前就被interrupt
* 在每次执行task前会lock，在中断前会tryLock
*/
private final class Worker extends AbstractQueuedSynchronizer implements Runnable {
final Thread thread;
Runnable firstTask;
volatile long completedTasks;
private Worker(Runnable task) {
setState(-1); //防止在runWorker前就被interrupts
this.firstTask = task;
this.thread = getThreadFactory().newThread(this);
}
@Override
public void run() {
runWorker(this);
}
protected boolean isHeldExclusively() {
return getState() != 0;
}
protected boolean tryAcquire(int unused) {
if (compareAndSetState(0,1)) {
setExclusiveOwnerThread(Thread.currentThread());
return true;
}
return false;
}
protected boolean tryRelease() {
setExclusiveOwnerThread(null);
setState(0);
return true;
}
public void lock() {
acquire(1);
}
public boolean tryLock() {
return tryAcquire(1);
}
public void unlock() {
release(1);
}
public boolean isLock() {
return isHeldExclusively();
}
void interruptIfStarted() {
Thread t;
// !=-1
if (getState() >= 0 && (t = thread) != null && !t.isInterrupted()) {
try {
t.interrupt();
} catch (SecurityException ignore) {
}
}
}
}
/**
* 1. 原来状态大于等于targetState
* 2. 设置targetState状态
* @param targetState SHUTDOWN or STOP
*/
private void advanceRunState(int targetState) {
for (;;) {
int c = ctl.get();
if (runStateAtLeast(c, targetState) ||
ctl.compareAndSet(c, ctlOf(targetState, workerCountOf(c))))
break;
}
}
void onShutdown() {
}
@Override
public void shutdown() {
final ReentrantLock mainLock = this.mainLock;
mainLock.lock();
try {
advanceRunState(SHUTDOWN);
interruptIdleWorkers(false);
onShutdown();
} finally {
mainLock.unlock();
}
tryTerminate();
}
private void interruptWorkers() {
final ReentrantLock mainLock = this.mainLock;
mainLock.lock();
try {
for (Worker w : workers) {
w.interruptIfStarted();
}
} finally {
mainLock.unlock();
}
}
private List<Runnable> drainQueue() {
BlockingQueue<Runnable> q = workQueue;
ArrayList<Runnable> taskList = new ArrayList<Runnable>();
q.drainTo(taskList);
if (!q.isEmpty()) {
for (Runnable r : q.toArray(new Runnable[0])) {
if (q.remove(r))
taskList.add(r);
}
}
return taskList;
}
public List<Runnable> shutdownNow() {
List<Runnable> tasks;
final ReentrantLock mainLock = this.mainLock;
mainLock.lock();
try {
advanceRunState(STOP);
interruptWorkers();
tasks = drainQueue();
} finally {
mainLock.unlock();
}
tryTerminate();
return tasks;
}
}
扫一扫
469
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
