Tomcat线程池分析

本文转载自：
作者：阿里巴巴大淘宝技术
链接：https://www.zhihu.com/question/336683528/answer/2518487120
来源：知乎

Tomcat请求处理过程

Tomcat 的整体架构包含连接器和容器两大部分，其中连接器负责与外部通信，容器负责内部逻辑处理。在连接器中：

1. 使用 ProtocolHandler 接口来封装I/O模型和应用层协议的差异，其中I/O模型可以选择非阻塞I/O、异步I/O或APR，应用层协议可以选择HTTP、HTTPS或AJP。ProtocolHandler将I/O模型和应用层协议进行组合，让EndPoint只负责字节流的收发，Processor负责将字节流解析为Tomcat Request/Response对象，实现功能模块的高内聚和低耦合，ProtocolHandler接口继承关系如下图示。
2. 通过适配器 Adapter 将Tomcat Request对象转换为标准的ServletRequest对象。

Tomcat为了实现请求的快速响应，使用线程池来提高请求的处理能力。
下面我们以HTTP非阻塞I/O为例对Tomcat线程池进行简要的分析。

1 Tomcat线程池创建

在Tomcat中，通过AbstractEndpoint类提供底层的网络I/O的处理，若用户没有配置自定义公共线程池，则AbstractEndpoint通过createExecutor方法来创建Tomcat默认线程池。

核心部分代码如下：

public void createExecutor() {
        internalExecutor = true;
        TaskQueue taskqueue = new TaskQueue();
        TaskThreadFactory tf = new TaskThreadFactory(getName() + "-exec-", daemon, getThreadPriority());
        executor = new ThreadPoolExecutor(getMinSpareThreads(), getMaxThreads(), 60, TimeUnit.SECONDS,taskqueue, tf);
        taskqueue.setParent( (ThreadPoolExecutor) executor);
}

其中，TaskQueue、ThreadPoolExecutor分别为Tomcat自定义任务队列、线程池实现。

2 Tomcat自定义ThreadPoolExecutor

Tomcat自定义线程池继承于java.util.concurrent.ThreadPoolExecutor，并新增了一些成员变量来更高效地统计已经提交但尚未完成的任务数量（submittedCount），包括已经在队列中的任务和已经交给工作线程但还未开始执行的任务。

/**
 * Same as a java.util.concurrent.ThreadPoolExecutor but implements a much more efficient
 * {@link #getSubmittedCount()} method, to be used to properly handle the work queue.
 * If a RejectedExecutionHandler is not specified a default one will be configured
 * and that one will always throw a RejectedExecutionException
 *
 */
public class ThreadPoolExecutor extends java.util.concurrent.ThreadPoolExecutor {

    /**
     * The number of tasks submitted but not yet finished. This includes tasks
     * in the queue and tasks that have been handed to a worker thread but the
     * latter did not start executing the task yet.
     * This number is always greater or equal to {@link #getActiveCount()}.
     */
    // 新增的submittedCount成员变量，用于统计已提交但还未完成的任务数
    private final AtomicInteger submittedCount = new AtomicInteger(0);
    private final AtomicLong lastContextStoppedTime = new AtomicLong(0L);
    // 构造函数
    public ThreadPoolExecutor(int corePoolSize, int maximumPoolSize, long keepAliveTime, TimeUnit unit, BlockingQueue<Runnable> workQueue, ThreadFactory threadFactory,
            RejectedExecutionHandler handler) {
        super(corePoolSize, maximumPoolSize, keepAliveTime, unit, workQueue, threadFactory, handler);
        // 预启动所有核心线程
        prestartAllCoreThreads();
    }

}

Tomcat在自定义线程池ThreadPoolExecutor中重写了execute()方法，并实现对提交执行的任务进行submittedCount加一。Tomcat在自定义ThreadPoolExecutor中，当线程池抛出RejectedExecutionException异常后，会调用force()方法再次向TaskQueue中进行添加任务的尝试。如果添加失败，则submittedCount减一后，再抛出RejectedExecutionException。

@Override
public void execute(Runnable command) {
    execute(command,0,TimeUnit.MILLISECONDS);
}

public void execute(Runnable command, long timeout, TimeUnit unit) {
    submittedCount.incrementAndGet();
    try {
        super.execute(command);
    } catch (RejectedExecutionException rx) {
        if (super.getQueue() instanceof TaskQueue) {
            final TaskQueue queue = (TaskQueue)super.getQueue();
            try {
                if (!queue.force(command, timeout, unit)) {
                    submittedCount.decrementAndGet();
                    throw new RejectedExecutionException("Queue capacity is full.");
                }
            } catch (InterruptedException x) {
                submittedCount.decrementAndGet();
                throw new RejectedExecutionException(x);
            }
        } else {
            submittedCount.decrementAndGet();
            throw rx;
        }

    }
}

3 Tomcat自定义任务队列

在Tomcat中重新定义了一个阻塞队列TaskQueue，它继承于LinkedBlockingQueue。在Tomcat中，核心线程数默认值为10，最大线程数默认为200，为了避免线程到达核心线程数后后续任务放入队列等待，Tomcat通过自定义任务队列TaskQueue重写offer方法实现了核心线程池数达到配置数后线程的创建。

具体地，从线程池任务调度机制实现可知，当offer方法返回false时，线程池将尝试创建新新线程，从而实现任务的快速响应。TaskQueue核心实现代码如下：

/**
 * As task queue specifically designed to run with a thread pool executor. The
 * task queue is optimised to properly utilize threads within a thread pool
 * executor. If you use a normal queue, the executor will spawn threads when
 * there are idle threads and you wont be able to force items onto the queue
 * itself.
 */
public class TaskQueue extends LinkedBlockingQueue<Runnable> {

    public boolean force(Runnable o, long timeout, TimeUnit unit) throws InterruptedException {
        if ( parent==null || parent.isShutdown() ) throw new RejectedExecutionException("Executor not running, can't force a command into the queue");
        return super.offer(o,timeout,unit); //forces the item onto the queue, to be used if the task is rejected
    }

    @Override
    public boolean offer(Runnable o) {
        // 1. parent为线程池，Tomcat中为自定义线程池实例
      //we can't do any checks
        if (parent==null) return super.offer(o);
        // 2. 当线程数达到最大线程数时，新提交任务入队
        //we are maxed out on threads, simply queue the object
        if (parent.getPoolSize() == parent.getMaximumPoolSize()) return super.offer(o);
        // 3. 当提交的任务数小于线程池中已有的线程数时，即有空闲线程，任务入队即可
        //we have idle threads, just add it to the queue
        if (parent.getSubmittedCount()<=(parent.getPoolSize())) return super.offer(o);
        // 4. 【关键点】如果当前线程数量未达到最大线程数，直接返回false，让线程池创建新线程
        //if we have less threads than maximum force creation of a new thread
        if (parent.getPoolSize()<parent.getMaximumPoolSize()) return false;
        // 5. 最后的兜底，放入队列
        //if we reached here, we need to add it to the queue
        return super.offer(o);
    }   
}

4 Tomcat自定义任务线程

Tomcat中通过自定义任务线程TaskThread实现对每个线程创建时间的记录；使用静态内部类WrappingRunnable对Runnable进行包装，用于对StopPooledThreadException异常类型的处理。

/**
 * A Thread implementation that records the time at which it was created.
 *
 */
public class TaskThread extends Thread {

    private final long creationTime;

    public TaskThread(ThreadGroup group, Runnable target, String name) {
        super(group, new WrappingRunnable(target), name);
        this.creationTime = System.currentTimeMillis();
    }


    /**
     * Wraps a {@link Runnable} to swallow any {@link StopPooledThreadException}
     * instead of letting it go and potentially trigger a break in a debugger.
     */
    private static class WrappingRunnable implements Runnable {
        private Runnable wrappedRunnable;
        WrappingRunnable(Runnable wrappedRunnable) {
            this.wrappedRunnable = wrappedRunnable;
        }
        @Override
        public void run() {
            try {
                wrappedRunnable.run();
            } catch(StopPooledThreadException exc) {
                //expected : we just swallow the exception to avoid disturbing
                //debuggers like eclipse's
                log.debug("Thread exiting on purpose", exc);
            }
        }

    }

}

思考&小结

1 Tomcat为什么要自定义线程池和任务队列实现？

JUC原生线程池在提交任务时，当工作线程数达到核心线程数后，继续提交任务会尝试将任务放入阻塞队列中，只有当前运行线程数未达到最大设定值且在任务队列任务满后，才会继续创建新的工作线程来处理任务，因此JUC原生线程池无法满足Tomcat快速响应的诉求。

2 Tomcat为什么使用无界队列？

Tomcat在EndPoint中通过acceptCount和maxConnections两个参数来避免过多请求积压。其中maxConnections为Tomcat在任意时刻接收和处理的最大连接数，当Tomcat接收的连接数达到maxConnections时，Acceptor不会读取accept队列中的连接；这时accept队列中的线程会一直阻塞着，直到Tomcat接收的连接数小于maxConnections（maxConnections默认为10000，如果设置为-1，则连接数不受限制）。acceptCount为accept队列的长度，当accept队列中连接的个数达到acceptCount时，即队列满，此时进来的请求一律被拒绝，默认值是100（基于Tomcat 8.5.43版本）。因此，通过acceptCount和maxConnections两个参数作用后，Tomcat默认的无界任务队列通常不会造成OOM。

/**
 * Allows the server developer to specify the acceptCount (backlog) that
 * should be used for server sockets. By default, this value
 * is 100.
 */
private int acceptCount = 100;

private int maxConnections = 10000;