上次我们分析到Processor线程把请求存入到了completedReceives队列里,我们这次课分析一下,请求存入到completedReceives以后接下来是如何处理的。
我们接着看Processor线程的run方法,前面我们已经分析了 configureNewConnections()方法和 poll()方法。接下来我们看看一下processCompletedReceives()方法:
private def processCompletedReceives() {
//TODO 针对completedReceives里的每个请求
selector.completedReceives.asScala.foreach { receive =>
try {
//TODO 从channels里面获取channel
val channel = selector.channel(receive.source)
val session = RequestChannel.Session(new KafkaPrincipal(KafkaPrincipal.USER_TYPE, channel.principal.getName),
channel.socketAddress)
//TODO 从channels里面获取channel
val req = RequestChannel.Request(processor = id, connectionId = receive.source, session = session, buffer = receive.payload, startTimeMs = time.milliseconds, securityProtocol = protocol)
//TODO 把请求存入requestQueue队列
requestChannel.sendRequest(req)
//TODO 移除OP_READ事件
selector.mute(receive.source)
} catch {
case e @ (_: InvalidRequestException | _: SchemaException) =>
// note that even though we got an exception, we can assume that receive.source is valid. Issues with constructing a valid receive object were handled earlier
error(s"Closing socket for ${receive.source} because of error", e)
close(selector, receive.source)
}
}
}
我们注意到里面有个重要的操作:把每个获取到的请求封装成了Request对象,然后存入到了RequestChannel的requestQueue队列里。
到此我们总结一下我们看到的所有Server端的网络架构:
其实到这一讲Kafka Server端这儿的网络架构就很清晰了,使用的是三层架构:Acceptor线程专门用来监听连接请求,如果有了连接请求就把请求交给Processor线程处理,每个线程拥有一个Selector。
每个Selector接受到请求以后,Processor线程把封装到Request的对象统一都存入了RequestChannel的队列。至于为什么这么设计如果没太理解的不着急,等我们把Server端的整个网络部分剖析完了会后我们再进行总结。
这节我们看到请求都到了RequestChannel的requestQueue队列里了,我们猜也能猜得到后面肯定会有线程去处理队列里面的请求,我们下一讲继续剖析。