参数定义
先来看看Tomcat的几个常见参数定义,如下
参数 | 设置方式 | 默认值 | 含义 |
---|---|---|---|
acceptCount | server.tomcat.accept-count | 100 | 接收队列,实质上是在操作系统已经完成三次握手等待accept的socket队列 |
maxConnections | server.tomcat.max-connections | 8192 | 最大连接数,实质上是指已经accept的socket数量 |
maxThreads | server.tomcat.threads.max | 200 | 用于处理已经accept的socket的线程最大数量 |
connectionTimeout | server.tomcat.connection-timeout | 60000ms | 是指socket连接后或读取报文字节过程中的超时时间 |
keepAliveTimeout | 实现WebServerFactoryCustomizer | 默认取connectionTimeout | 在同一个socket连接,处理完一次http请求后,等待下一次http请求的超时时间 |
MaxKeepAliveRequests | 实现WebServerFactoryCustomizer | 100 | 在同一个socket连接,最大处理http请求数量,达到数量后关闭socket |
测试代码
服务端是一个SpringBoot的应用,测试代码就是一个sleep5秒的方法,如下
@PostMapping("/save")
public String save(@RequestBody String body) throws InterruptedException {
Thread.sleep(5000);
return "success";
}
客户端测试代码,采用SocketHttpClient发送请求,如下
static ExecutorService getExecutorService() {
ThreadPoolExecutor threadPoolExecutor = new ThreadPoolExecutor(10, 100, 60, TimeUnit.SECONDS,
new ArrayBlockingQueue<Runnable>(1), r -> {
Thread thread = new Thread(r);
thread.setDaemon(true);
thread.setName("自定义线程池 " + RandomUtils.nextInt());
return thread;
});
return threadPoolExecutor;
}
public static void main(String[] args) throws InterruptedException {
ExecutorService executorService = getExecutorService();
SocketHttpRequest socketHttpRequest = new SocketHttpRequest();
socketHttpRequest.setHost("127.0.0.1");
socketHttpRequest.setPort(9100);
socketHttpRequest.setMethod(HttpMethod.POST.name());
socketHttpRequest.setUrl("/test/save");
socketHttpRequest.setBody("test");
final CopyOnWriteArrayList<SendRecord> sendRecordList = new CopyOnWriteArrayList<>();
int n = 2;
for (int i = 1; i <= n; i++) {
SendRecord sendRecord = new SendRecord();
sendRecord.setId(i);
int a = i;
Thread.sleep(200);
executorService.execute(() -> {
try {
sendRecord.setConnectTime(new Date());
SocketHttpClient socketHttpClient = new SocketHttpClient("127.0.0.1", 9100);
//Thread.sleep( a*4000);
sendRecord.setSendTime(new Date());
SocketHttpResponse socketHttpResponse = socketHttpClient.sendRequest(socketHttpRequest);
sendRecord.setResult(socketHttpResponse.getBody());
// try {
// Thread.sleep(1000);
// socketHttpResponse = socketHttpClient.sendRequest(socketHttpRequest);
// System.out.println("同一socket连接,间隔1秒后第二次请求结果:" + socketHttpResponse.getBody());
// } catch (Exception ex) {
// System.out.println("同一socket连接,间隔1秒后第二次请求结果:" + ex.getMessage());
// }
socketHttpClient.close();
} catch (Exception e) {
sendRecord.setResult(e.getMessage());
} finally {
sendRecord.setEndTime(new Date());
sendRecordList.add(sendRecord);
}
});
}
while (sendRecordList.size() < n) {
Thread.sleep(1000);
}
sendRecordList.stream().sorted(Comparator.comparing(x -> x.getId())).forEach(x -> {
System.out.println(String.format("id:%s,connectTime:%s,sendTime:%s,endTime:%s,result:%s", x.getId()
, DateFormatUtils.format(x.getConnectTime(), "HH:mm:ss.ss")
, x.getSendTime() == null ? null : DateFormatUtils.format(x.getSendTime(), "HH:mm:ss.ss")
, x.getEndTime() == null ? null : DateFormatUtils.format(x.getEndTime(), "HH:mm:ss.ss"), x.getResult()));
});
}
acceptCount和maxConnections
测试1
发送5次请求,参数如下
server.tomcat.accept-count=1
server.tomcat.max-connections=1
server.tomcat.threads.max=100
id:1,connectTime:10:27:18.18,sendTime:10:27:18.18,endTime:10:27:23.23,result:success
id:2,connectTime:10:27:18.18,sendTime:10:27:18.18,endTime:10:27:28.28,result:success
id:3,connectTime:10:27:18.18,sendTime:null,endTime:10:27:20.20,result:Connection refused: connect
id:4,connectTime:10:27:18.18,sendTime:null,endTime:10:27:20.20,result:Connection refused: connect
id:5,connectTime:10:27:18.18,sendTime:null,endTime:10:27:21.21,result:Connection refused: connect
第一到达服务端的请求是id1,这时已经达到最大连接数,id2只能在accept队列等待,这时accept队列也满了,所以后面三次请求连接被操作系统拒绝了
测试2
发送5次请求,参数如下
server.tomcat.accept-count=1
server.tomcat.max-connections=2
server.tomcat.threads.max=100
id:1,connectTime:10:29:48.48,sendTime:10:29:49.49,endTime:10:29:54.54,result:success
id:2,connectTime:10:29:49.49,sendTime:10:29:49.49,endTime:10:29:54.54,result:success
id:3,connectTime:10:29:49.49,sendTime:10:29:49.49,endTime:10:29:59.59,result:success
id:4,connectTime:10:29:49.49,sendTime:null,endTime:10:29:51.51,result:Connection refused: connect
id:5,connectTime:10:29:49.49,sendTime:null,endTime:10:29:51.51,result:Connection refused: connect
id1和id2的处理完成时间相同,说明同时都在connection队列中被处理,id3处理完成时间在5秒后,说明在accept队列等待了5秒才进入connection队列,然后剩下两个请求因为accept和onnection队列都满了,所以都拒绝了
测试3
发送5次请求,参数如下
server.tomcat.accept-count=2
server.tomcat.max-connections=2
server.tomcat.threads.max=100
id:1,connectTime:10:31:38.38,sendTime:10:31:39.39,endTime:10:31:44.44,result:success
id:2,connectTime:10:31:39.39,sendTime:10:31:39.39,endTime:10:31:44.44,result:success
id:3,connectTime:10:31:39.39,sendTime:10:31:39.39,endTime:10:31:49.49,result:success
id:4,connectTime:10:31:39.39,sendTime:10:31:39.39,endTime:10:31:49.49,result:success
id:5,connectTime:10:31:39.39,sendTime:null,endTime:10:31:41.41,result:Connection refused: connect
相比测试2,多了id4在accept队列等待,符合预期
测试4
并发发送5次请求,参数如下
server.tomcat.accept-count=1
server.tomcat.max-connections=1
server.tomcat.threads.max=100
id:1,connectTime:10:54:18.18,sendTime:10:54:18.18,endTime:10:54:43.43,result:success
id:2,connectTime:10:54:18.18,sendTime:10:54:18.18,endTime:10:54:23.23,result:success
id:3,connectTime:10:54:18.18,sendTime:10:54:18.18,endTime:10:54:33.33,result:success
id:4,connectTime:10:54:18.18,sendTime:10:54:18.18,endTime:10:54:28.28,result:success
id:5,connectTime:10:54:18.18,sendTime:10:54:18.18,endTime:10:54:38.38,result:success
相比测试1,改成并发发送五次请求,按道理应该有三次是连接拒绝的,但是实测结果都成功了,查看处理结束时间都是间隔5秒,说明同一时间只有一个请求在connection队列,是accept队列没控制住了,五次请求都被accept队列接收了
maxThreads
测试5
发送5次请求,参数如下
server.tomcat.accept-count=2
server.tomcat.max-connections=2
server.tomcat.threads.max=1
id:1,connectTime:10:35:43.43,sendTime:10:35:43.43,endTime:10:35:48.48,result:success
id:2,connectTime:10:35:43.43,sendTime:10:35:43.43,endTime:10:35:53.53,result:success
id:3,connectTime:10:35:43.43,sendTime:10:35:43.43,endTime:10:35:58.58,result:success
id:4,connectTime:10:35:43.43,sendTime:10:35:43.43,endTime:10:36:03.03,result:success
id:5,connectTime:10:35:44.44,sendTime:null,endTime:10:35:46.46,result:Connection refused: connect
相比测试3,最大处理线程变成1,所以id1~id4的处理完成时间依次延后5秒,符合预期
connectionTimeout
测试6
server.tomcat.connection-timeout=5000
server.tomcat.accept-count=1
server.tomcat.max-connections=1
server.tomcat.threads.max=100
客户端增加代码
SocketHttpClient socketHttpClient = new SocketHttpClient("127.0.0.1", 9100);
Thread.sleep(4000);
id:1,connectTime:11:16:17.17,sendTime:11:16:21.21,endTime:11:16:26.26,result:success
socket连接成功后,4秒后才发送请求,响应成功,符合预期
测试7
server.tomcat.connection-timeout=5000
server.tomcat.accept-count=1
server.tomcat.max-connections=1
server.tomcat.threads.max=100
客户端增加代码
SocketHttpClient socketHttpClient = new SocketHttpClient("127.0.0.1", 9100);
Thread.sleep(7000);
id:1,connectTime:11:15:47.47,sendTime:11:15:54.54,endTime:11:15:54.54,result:Software caused connection abort: socket write error
socket连接成功后,7秒后才发送报文,超过设置connection-timeout=5000,发生socket异常,说明服务端的socket连接读取超时关闭了,符合预期
测试8
SocketHttpClient修改代码
//添加请求行
headStringBuffer.append(socketHttpRequest.getMethod()).append(" ").append(socketHttpRequest.getUrl()).append(" ").append("HTTP/1.1").append("\r\n");
out.write(headStringBuffer.toString());
out.flush();
Thread.sleep(7000);
StringBuffer sb = new StringBuffer();
相比测试7,写入请求行后,延迟7秒后才写请求头,也发生socket异常.这就说明无论在连接成功后延迟,还是传输报文的过程中有延迟,都是会触发服务端的connectTimeout异常
测试9
SocketHttpClient socketHttpClient = new SocketHttpClient("127.0.0.1", 9100);
Thread.sleep(a*4000);
id:1,connectTime:14:20:09.09,sendTime:14:20:13.13,endTime:14:20:18.18,result:success
id:2,connectTime:14:20:09.09,sendTime:14:20:17.17,endTime:14:20:23.23,result:success
这次测试连续发送两次请求,第二次socket连接成功8秒后才发送报文,但是也成功响应了,看似不符合connection-timeout=5000的限制,但是要看id1的结束时间是14:20:18,在这之前id2一直在accept队列等待,并没有在connection队列.所以connectionTimeout的计算规则应为socket进入connection队列才开始计时.
keepAliveTimeout
测试10
protocol.setKeepAliveTimeout(10000);
sendRecord.setResult(socketHttpResponse.getBody());
try {
Thread.sleep(7000);
socketHttpResponse = socketHttpClient.sendRequest(socketHttpRequest);
System.out.println("同一socket连接,间隔7秒后第二次请求结果:" + socketHttpResponse.getBody());
} catch (Exception ex) {
System.out.println("同一socket连接,间隔7秒后第二次请求结果:" + ex.getMessage());
}
同一socket连接,间隔7秒后第二次请求结果:success
id:1,connectTime:17:04:17.17,sendTime:17:04:17.17,endTime:17:04:34.34,result:success
同一socket,第二次请求延迟7秒,小于KeepAliveTimeout的10秒,响应正常,符合预期
测试11
protocol.setKeepAliveTimeout(10000);
sendRecord.setResult(socketHttpResponse.getBody());
try {
Thread.sleep(12000);
socketHttpResponse = socketHttpClient.sendRequest(socketHttpRequest);
System.out.println("同一socket连接,间隔12秒后第二次请求结果:" + socketHttpResponse.getBody());
} catch (Exception ex) {
System.out.println("同一socket连接,间隔12秒后第二次请求结果:" + ex.getMessage());
}
同一socket连接,间隔12秒后第二次请求结果:Software caused connection abort: socket write error
id:1,connectTime:17:04:52.52,sendTime:17:04:52.52,endTime:17:05:09.09,result:success
同一socket,第二次请求延迟12秒,大于KeepAliveTimeout的10秒,socket写入失败,也符合预期
MaxKeepAliveRequests
测试12
protocol.setMaxKeepAliveRequests(1);
try {
Thread.sleep(1000);
socketHttpResponse = socketHttpClient.sendRequest(socketHttpRequest);
System.out.println("同一socket连接,间隔1秒后第二次请求结果:" + socketHttpResponse.getBody());
} catch (Exception ex) {
System.out.println("同一socket连接,间隔1秒后第二次请求结果:" + ex.getMessage());
}
同一socket连接,间隔1秒后第二次请求结果:Software caused connection abort: socket write error
同一socket,第二次请求失败,大于MaxKeepAliveRequests的1次,socket写入失败,符合预期
测试13
protocol.setMaxKeepAliveRequests(2);
try {
Thread.sleep(1000);
socketHttpResponse = socketHttpClient.sendRequest(socketHttpRequest);
System.out.println("同一socket连接,间隔1秒后第二次请求结果:" + socketHttpResponse.getBody());
} catch (Exception ex) {
System.out.println("同一socket连接,间隔1秒后第二次请求结果:" + ex.getMessage());
}
同一socket连接,间隔1秒后第二次请求结果:success
相比测试12,MaxKeepAliveRequests改成2,响应成功,符合预期
源码简析
上面测试都是基于客户端,对服务端进行黑盒测试的,现在我们来看看那几个参数在Tomcat的源码中哪里.
acceptCount
,相关代码可以查看org.apache.tomcat.util.net.NioEndpoint.initServerSocket(),它决定了Socket中连接队列backlog的值,关于backlog,不同操作系统有不同的实现,上面测试5是在window环境上的,可能linux结果可能会有所不同maxConnections
,相关代码可以查看org.apache.tomcat.util.net.Acceptor.run 和org.apache.tomcat.util.net.AbstractEndpoint.countUpOrAwaitConnection,Tomcat开启了一个accept线程,通过countUpOrAwaitConnection方法判断是否达到maxConnection,然后从serverSocketChannel中获取连接,放在org.apache.tomcat.util.net.AbstractEndpoint.connections列表中maxThreads
,相关代码可以查看org.apache.tomcat.util.net.AbstractEndpoint.setMaxThreads,Tomcat创建了最大线程为maxThreads的线程池executor.另外,Tomcat也启动了org.apache.tomcat.util.net.NioEndpoint.Poller的线程,负责将已经accept的socket封装在org.apache.tomcat.util.net.NioEndpoint.SocketProcessor,指派给executor线程池处理connectionTimeout
和keepAliveTimeout
相关代码可以查看org.apache.tomcat.util.net.NioEndpoint.Poller.timeout方法,这个方法里面不断轮询所有注册到Poller的socketChannel,如果最后一次读取时间到当前时间大于timeout时间,则结束socket连接.这个timeout可能是connectionTimeout和keepAliveTimeout,具体是读取哪个值,要看字节读取阶段,相关逻辑在org.apache.coyote.http11.Http11Processor.service方法中MaxKeepAliveRequests
,也是在org.apache.coyote.http11.Http11Processor.service方法,当请求达到MaxKeepAliveRequests的值时,会将keepAlive改为flase,然后退出读取当前socket的循环
简单来说,Tomcat启动的时候,会实例化ServerSocketChannel监听端口,创建一个线程池executor和启动两个线程ClientPoller和Acceptor,Acceptor负责将连接成功的tcp连接取出,注册到ClientPoller中,ClientPoller负责轮询所有channel,当有可读事件,就将可读的socketChannel转交给executor处理,executor会分配一个名为xx-exec-xx的线程去处理,这exec线程主要是从SocketChannel中读取字节,解析成HTTP报文,封装成Request传递下去,后面的链路主要就是我们比较熟悉的Filter链,DispatcherServlet和Controller了