Acceptor
监听
每个acceptor绑定一个ServerSocketChannel通道
- ServerConnector由其管理容器启动start
- 打开连接:打开ServerSocketChannel通道,并配置accept通道(_acceptChannel)为阻塞模式,绑定地址端口
- 构建org.eclipse.jetty.server.AbstractConnector.Acceptor并提交至线程池(org.eclipse.jetty.server.Server#_threadPool,默认大小200=org.eclipse.jetty.util.thread.QueuedThreadPool#QueuedThreadPool())执行启动,如果通道已打开,阻塞在accept接受客户端请求,等待数据,线程池中异步执行
- 如果存在多个Acceptors则配置accept通道(_acceptChannel=ServerSocketChannel)为非阻塞模式
- 选择器管理器提交org.eclipse.jetty.io.ManagedSelector.Acceptor(绑定SocketChannel、attachment为null)至队列并唤醒选择器通知选择器存在更新选择器更新
- 选择器生产者自旋阻塞在java.nio.channels.Selector#select()等待唤醒并处理更新
- 处理选择器更新,回调选择器更新实例的org.eclipse.jetty.io.ManagedSelector.Acceptor#update方法,将SelectableChannel(accept通道(_acceptChannel=ServerSocketChannel))注册至选择器Selector
数据到达服务端
客户端数据到达服务端,阻塞唤醒
- 服务端接收到数据唤醒accept阻塞
- 配置SocketChannel通道为非阻塞模式
- 选择器管理器提交org.eclipse.jetty.io.ManagedSelector.Accept(绑定SocketChannel、attachment为null)至队列并唤醒选择器通知选择器存在更新选择器更新
- 选择器生产者自旋阻塞在java.nio.channels.Selector#select()等待唤醒并处理更新
- 处理选择器更新,回调选择器更新实例的org.eclipse.jetty.io.ManagedSelector.Accept#update方法,将SelectableChannel(与客户端建立的数据通道SocketChannel)注册至选择器Selector
jetty线程模型
总结
Acceptor可以理解为NIO的:java.nio.channels.ServerSocketChannel
Selector可以理解为NIO的:java.nio.channels.Selector
线程池类型
Jetty默认线程池为:org.eclipse.jetty.util.thread.QueuedThreadPool,线程池资源监控:org.eclipse.jetty.server.LowResourceMonitor.LowResourceCheck(例如:org.eclipse.jetty.server.LowResourceMonitor.MainThreadPoolLowResourceCheck#isLowOnResources,org.eclipse.jetty.server.LowResourceMonitor.ConnectorsThreadPoolLowResourceCheck#isLowOnResources)
线程池资源监控处理
线程池处于高水位线时间超出最大值:org.eclipse.jetty.server.LowResourceMonitor#_maxLowResourcesTime,默认0,则将所有链接设置为拒绝接收数据:org.eclipse.jetty.server.LowResourceMonitor#setLowResources
线程池低于高水位线时恢复接收数据:org.eclipse.jetty.server.LowResourceMonitor#clearLowResources
监控
public class MyJettyServerCustomizer implements JettyServerCustomizer {
@Override
public void customize(Server server) {
LowResourceMonitor lowResourceMonitor = new LowResourceMonitor(server);
LowResourceCheck lowResourceCheck = lowResourceMonitor.new MainThreadPoolLowResourceCheck();
lowResourceMonitor.addLowResourceCheck(lowResourceCheck);
server.addBean(lowResourceMonitor);
}
}
配置监控
@Bean
public ServletWebServerFactory servletContainer() {
JettyServletWebServerFactory jetty = new JettyServletWebServerFactory();
QueuedThreadPool queuedThreadPool = new QueuedThreadPool(4,4);
jetty.setThreadPool(queuedThreadPool);
jetty.setPort(Integer.parseInt(serverPort.trim()));
MyJettyServerCustomizer myJettyServerCustomizer = new MyJettyServerCustomizer();
jetty.addServerCustomizers(myJettyServerCustomizer);
return jetty;
}
启用低资源时拒绝服务
public class MyJettyServerCustomizer implements JettyServerCustomizer {
@Override
public void customize(Server server) {
LowResourceMonitor lowResourceMonitor = new LowResourceMonitor(server);
// 启用低资源拒绝服务
lowResourceMonitor.setAcceptingInLowResources(false);
// 低资源持续时间达到阈值拒绝服务
lowResourceMonitor.setMaxLowResourcesTime(1);
LowResourceCheck lowResourceCheck = lowResourceMonitor.new MainThreadPoolLowResourceCheck();
lowResourceMonitor.addLowResourceCheck(lowResourceCheck);
server.addBean(lowResourceMonitor);
}
}
启用后可能在资源不足时拒绝服务,出现报错
客户端错误信息
org.apache.http.NoHttpResponseException: localhost:8000 failed to respond
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:141)
at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:56)
at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:259)
at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:163)
at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:157)
at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:273)
at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:125)
at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:272)
at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:186)
at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:89)
at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:110)
at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:185)
at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:83)
at org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl.executeRequest(HTTPHC4Impl.java:939)
at org.apache.jmeter.protocol.http.sampler.HTTPHC4Impl.sample(HTTPHC4Impl.java:650)
at org.apache.jmeter.protocol.http.sampler.HTTPSamplerProxy.sample(HTTPSamplerProxy.java:66)
at org.apache.jmeter.protocol.http.sampler.HTTPSamplerBase.sample(HTTPSamplerBase.java:1301)
at org.apache.jmeter.protocol.http.sampler.HTTPSamplerBase.sample(HTTPSamplerBase.java:1290)
at org.apache.jmeter.threads.JMeterThread.doSampling(JMeterThread.java:651)
at org.apache.jmeter.threads.JMeterThread.executeSamplePackage(JMeterThread.java:570)
at org.apache.jmeter.threads.JMeterThread.processSampler(JMeterThread.java:501)
at org.apache.jmeter.threads.JMeterThread.run(JMeterThread.java:268)
at java.base/java.lang.Thread.run(Thread.java:833)
服务端错误信息
2023-03-05 10:55:49,918 WARN [qtp1493142905-23] o.e.j.u.t.QueuedThreadPool:execute:538 {} QueuedThreadPool[qtp1493142905]@58ff8d79{STARTED,4<=4<=4,i=0,r=1,q=0}[ReservedThreadExecutor@6f79390f{s=0/1,p=1}] rejected org.eclipse.jetty.io.ManagedSelector$Accept@640d906c
2023-03-05 10:55:49,919 WARN [qtp1493142905-23] o.e.j.u.t.QueuedThreadPool:execute:538 {} QueuedThreadPool[qtp1493142905]@58ff8d79{STARTED,4<=4<=4,i=1,r=1,q=0}[ReservedThreadExecutor@6f79390f{s=0/1,p=1}] rejected CEP:SocketChannelEndPoint@19cfb8d{/1.0.0.0:21363<->/[0:0:0:0:0:0:7f00:1]:8000,OPEN,fill=FI,flush=-,to=1/30000}{io=1/0,kio=1,kro=1}->HttpConnection@64db9962[p=HttpParser{s=START,0 of -1},g=HttpGenerator@6c1819f3{s=START}]=>HttpChannelOverHttp@3b35730{s=HttpChannelState@b3529da{s=IDLE rs=BLOCKING os=OPEN is=IDLE awp=false se=false i=true al=0},r=0,c=false/false,a=IDLE,uri=null,age=0}:runFillable:BLOCKING
2023-03-05 10:55:49,919 WARN [qtp1493142905-23] o.e.j.u.t.s.EatWhatYouKill:execute:378 {}
java.util.concurrent.RejectedExecutionException: CEP:SocketChannelEndPoint@19cfb8d{/1.0.0.0:21363<->/[0:0:0:0:0:0:7f00:1]:8000,OPEN,fill=FI,flush=-,to=1/30000}{io=1/0,kio=1,kro=1}->HttpConnection@64db9962[p=HttpParser{s=START,0 of -1},g=HttpGenerator@6c1819f3{s=START}]=>HttpChannelOverHttp@3b35730{s=HttpChannelState@b3529da{s=IDLE rs=BLOCKING os=OPEN is=IDLE awp=false se=false i=true al=0},r=0,c=false/false,a=IDLE,uri=null,age=0}:runFillable:BLOCKING
at org.eclipse.jetty.util.thread.QueuedThreadPool.execute(QueuedThreadPool.java:539) ~[jetty-util-9.4.22.v20191022.jar:9.4.22.v20191022]
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.execute(EatWhatYouKill.java:373) [jetty-util-9.4.22.v20191022.jar:9.4.22.v20191022]
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:308) [jetty-util-9.4.22.v20191022.jar:9.4.22.v20191022]
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:171) [jetty-util-9.4.22.v20191022.jar:9.4.22.v20191022]
at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:129) [jetty-util-9.4.22.v20191022.jar:9.4.22.v20191022]
at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:388) [jetty-util-9.4.22.v20191022.jar:9.4.22.v20191022]
at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:806) [jetty-util-9.4.22.v20191022.jar:9.4.22.v20191022]
at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:938) [jetty-util-9.4.22.v20191022.jar:9.4.22.v20191022]
at java.lang.Thread.run(Thread.java:833) [?:?]
线程池
- 默认线程池:org.eclipse.jetty.util.thread.QueuedThreadPool,min=8,max=200
- 队列:org.eclipse.jetty.util.BlockingArrayQueue#BlockingArrayQueue(int, int),capacity=grawBy=Math.max(_minThreads, 8) * 1024
- 拒绝策略:直接抛异常throw new RejectedExecutionException(job.toString());
- 如果开启了监控LowResourceMonitor,监控在检测到系统繁忙时会暂停数据处理,不再繁忙时恢复数据处理
- 注意如果线程池类型为QueuedThreadPool配置会被重写,需要结合jetty配置,重写逻辑设计配置可以参考源码:org.springframework.boot.autoconfigure.web.embedded.JettyWebServerFactoryCustomizer#customize
优点
架构设计的优点很明显,并发扩展能力很强大,可以支持多个Connector的垂直向扩展,由于使用了多路复用IO,每个Selector可以监听多个Channel,进一步提升了框架的并发能力
缺点
线程模型较为单调,对于多个Connector其实可以理解为多个租户,如果对多个租户做到线程池的资源隔离的话个人感觉效果会更好。不过其实也还好,算是鸡蛋里挑骨头了,毕竟多Connector的场景以及多租户的场景可以通过水平方向的扩展+DNS路由来弥补