tomcat线程模型

入口方法

if (getPort() < 0) {
    throw new LifecycleException(sm.getString(
            "coyoteConnector.invalidPort", Integer.valueOf(getPort())));
}

setState(LifecycleState.STARTING);

try {
    // 具体的协议处理方法
    protocolHandler.start();
} catch (Exception e) {
    throw new LifecycleException(
            sm.getString("coyoteConnector.protocolHandlerStartFailed"), e);
}

线程模型之acceptor

方法栈：protocolHandler.start()

-> org.apache.tomcat.util.net.AbstractEndpoint#start

->org.apache.tomcat.util.net.NioEndpoint#bind 生成serversocket

->org.apache.tomcat.util.net.NioEndpoint#startInternal 启动acceptor

->org.apache.tomcat.util.net.AbstractEndpoint#startAcceptorThreads

serverSock = AsynchronousServerSocketChannel.open(threadGroup);
socketProperties.setProperties(serverSock);
InetSocketAddress addr = (getAddress()!=null?new InetSocketAddress(getAddress(),getPort()):new InetSocketAddress(getPort()));
serverSock.bind(addr,getAcceptCount());

protected final void startAcceptorThreads() {
    // 获取acceptor的线程数量， 默认是1
    int count = getAcceptorThreadCount();
    // new acceptor
    acceptors = new Acceptor[count];

    for (int i = 0; i < count; i++) {
        // 其实现了runnable接口，用新线程包装并且启动。
        acceptors[i] = createAcceptor();
        String threadName = getName() + "-Acceptor-" + i;
        acceptors[i].setThreadName(threadName);
        Thread t = new Thread(acceptors[i], threadName);
        t.setPriority(getAcceptorThreadPriority());
        t.setDaemon(getDaemon());
        t.start();
    }
}

//这个地方表示需要一个连接，如果连接达到上线，则再次wait
countUpOrAwaitConnection();

AsynchronousSocketChannel socket = null;
try {
    // 获取客户端的连接socket
    socket = serverSock.accept().get();
} catch (Exception e) {
    // 生成socket失败，则弃用已经占据的连接
    countDownConnection();
}

// Configure the socket
if (running && !paused) {
    // 处理新生成的socket，这个方法比较重要。是socket处理的关键转折点
    if (!setSocketOptions(socket)) {
        closeSocket(socket);
   }
} else {
    closeSocket(socket);
}

// 设置为非阻塞的
socket.configureBlocking(false);
Socket sock = socket.socket();
// 设置属性，缓冲区等
socketProperties.setProperties(sock);

NioChannel channel = nioChannels.pop();
if (channel == null) {
    SocketBufferHandler bufhandler = new SocketBufferHandler(
            socketProperties.getAppReadBufSize(),
            socketProperties.getAppWriteBufSize(),
            socketProperties.getDirectBuffer());
    if (isSSLEnabled()) {
        channel = new SecureNioChannel(socket, bufhandler, selectorPool, this);
    } else {
        channel = new NioChannel(socket, bufhandler);
    }
} else {
    channel.setIOChannel(socket);
    channel.reset();
}

getPoller0().register(channel);

// 线程池轮询算法
public Poller getPoller0() {
    int idx = Math.abs(pollerRotater.incrementAndGet()) % pollers.length;
    return pollers[idx];
}

// 对socket进行属性设置，以及生成一个poller事件，交给poller执行。
public void register(final NioChannel socket) {
    socket.setPoller(this);
    NioSocketWrapper ka = new NioSocketWrapper(socket, NioEndpoint.this);
    socket.setSocketWrapper(ka);
    ka.setPoller(this);
    ka.setReadTimeout(getSocketProperties().getSoTimeout());
    ka.setWriteTimeout(getSocketProperties().getSoTimeout());
    ka.setKeepAliveLeft(NioEndpoint.this.getMaxKeepAliveRequests());
    ka.setSecure(isSSLEnabled());
    ka.setReadTimeout(getConnectionTimeout());
    ka.setWriteTimeout(getConnectionTimeout());
    PollerEvent r = eventCache.pop();
    ka.interestOps(SelectionKey.OP_READ);//this is what OP_REGISTER turns into.
    if ( r==null) r = new PollerEvent(socket,ka,OP_REGISTER);
    else r.reset(socket,ka,OP_REGISTER);
    // 转换成pollerevent，添加到队列里面，等待poller线程执行
    addEvent(r);
}

线程模型之poller

poller线程是在acceptor与worker之间的线程。其主要作用是接收acceptor的线程，然后注册读写事件，在读写时间来的时候将其转交给worker线程。

方法栈：protocolHandler.start()

-> org.apache.tomcat.util.net.AbstractEndpoint#start

->org.apache.tomcat.util.net.NioEndpoint#bind 生成serversocket

->org.apache.tomcat.util.net.NioEndpoint#startInternal 启动acceptor

->org.apache.tomcat.util.net.AbstractEndpoint#new Poller new线程，然后启动

// events方法会遍历队列中的所有event，然后调用其run方法执行
hasEvents = events();
if (wakeupCounter.getAndSet(-1) > 0) {
    //if we are here, means we have other stuff to do
    //do a non blocking select
    keyCount = selector.selectNow();
} else {
    keyCount = selector.select(selectorTimeout);
}
wakeupCounter.set(0);

// 迭代获取select事件
Iterator<SelectionKey> iterator =
    keyCount > 0 ? selector.selectedKeys().iterator() : null;
// Walk through the collection of ready keys and dispatch
// any active event.
while (iterator != null && iterator.hasNext()) {
    SelectionKey sk = iterator.next();
    NioSocketWrapper attachment = (NioSocketWrapper)sk.attachment();
    // Attachment may be null if another thread has called
    // cancelledKey()
    if (attachment == null) {
        iterator.remove();
    } else {
        iterator.remove();
        processKey(sk, attachment);
    }
}

这个方法是在poller线程中执行events的时候调用的，任务的添加是在acceptor到socket之后，把其注册到了这个poller对应的socket上。

if (interestOps == OP_REGISTER) {
    try {
        // 把socket注册到selector上，并且设置监听读事件
        socket.getIOChannel().register(
                socket.getPoller().getSelector(), SelectionKey.OP_READ, socketWrapper);
    } catch (Exception x) {
        log.error(sm.getString("endpoint.nio.registerFail"), x);
    }
}

这个processKey方法也是在selector在select之后，对事件处理的核心方法

if (sk.isReadable()) {
    if (!processSocket(attachment, SocketEvent.OPEN_READ, true)) {
        closeSocket = true;
    }
}
if (!closeSocket && sk.isWritable()) {
    if (!processSocket(attachment, SocketEvent.OPEN_WRITE, true)) {
        closeSocket = true;
    }
}

线程模型之worker

在上述方法栈的基础上，又启动了worker线程

org.apache.tomcat.util.net.AbstractEndpoint#startAcceptorThreads

-> org.apache.tomcat.util.net.NioEndpoint#setSocketOptions

-> processSocket(socketWrapper, SocketEvent.OPEN_READ, true)

if (socketWrapper == null) {
      return false;
  }
  SocketProcessorBase<S> sc = processorCache.pop();
  if (sc == null) {
      sc = createSocketProcessor(socketWrapper, event);
  } else {
      sc.reset(socketWrapper, event);
  }
  Executor executor = getExecutor();
  // 这个是转折到具体的worker线程池的关键点，提交任务至worker线程池
  if (dispatch && executor != null) {
      executor.execute(sc);
  } else {
      sc.run();
  }

接下来就会在工作线程池中调用processor的具体处理方法。包括常用的servlet、filter、interceptor、session管理等机制都在这个内部。

connection控制

方法栈：protocolHandler.start()

-> org.apache.tomcat.util.net.AbstractEndpoint#start

->org.apache.tomcat.util.net.NioEndpoint#startInternal 启动acceptor

-> org.apache.tomcat.util.net.AbstractEndpoint#initializeConnectionLatch

// 使用LimitLatch来控制最大连接数
protected LimitLatch initializeConnectionLatch() {
    if (maxConnections==-1) return null;
    if (connectionLimitLatch==null) {
        connectionLimitLatch = new LimitLatch(getMaxConnections());
    }
    return connectionLimitLatch;
}

// 先判断连接数还够用不，不够用的话需要等待
countUpOrAwaitConnection();

SocketChannel socket = null;
try {
    // Accept the next incoming connection from the server
    // socket
    socket = serverSock.accept();
} catch (IOException ioe) {
    // We didn't get a socket
    // 连接建立遇到了异常，则把占用的连接数给释放掉
    countDownConnection();
    if (running) {
        // Introduce delay if necessary
        errorDelay = handleExceptionWithDelay(errorDelay);
        // re-throw
        throw ioe;
    } else {
        break;
    }
}

tomcat的常用调优参数

• 在server.xml中修改<Connector>标签的connectionTimeout、maxThreads、minSpareThreads、acceptCount等属性。
• connectionTimeout 设置连接超时时间。
• maxThreads 设置Tomcat可以处理的最大并发线程数。
• minSpareThreads 设置核心线程数
• acceptCount 设置允许的最大连接数，超过这个数的连接将会等待。

executor = new ThreadPoolExecutor(getMinSpareThreads(), getMaxThreads(), 60, TimeUnit.SECONDS,taskqueue, tf);

this.maxThreads = maxThreads;
Executor executor = this.executor;
if (internalExecutor && executor instanceof java.util.concurrent.ThreadPoolExecutor) {
    // The internal executor should always be an instance of
    // j.u.c.ThreadPoolExecutor but it may be null if the endpoint is
    // not running.
    // This check also avoids various threading issues.
    ((java.util.concurrent.ThreadPoolExecutor) executor).setMaximumPoolSize(maxThreads);
}