kafka源码阅读系列之三（socketServer组件和NIO）

顾名思义，socketServer管理的是网络连接相关的任务
值得一提的是，kafka使用NIO模型来管理所有的网络连接任务，以期提升处理性能

/**
 * An NIO socket server. The threading model is
 *   1 Acceptor thread that handles new connections
 *   Acceptor has N Processor threads that each have their own selector and read requests from sockets
 *   M Handler threads that handle requests and produce responses back to the processor threads for writing.
 */
class SocketServer(val config: KafkaConfig, val metrics: Metrics, val time: Time) extends Logging with KafkaMetricsGroup

注释上说的很明白，一个NIO的socket服务器
其线程模型是一个Acceptor 接收器线程负责分发连接，N个processor处理器拥有各自的selector管理器来读取socket任务请求，同时M个handler线程负责真正处理任务请求，并将响应返回给各自的processor来进行数据回写

一个典型的reactor 同步非阻塞多路io复用模型
来看看socketServer是如何实现的

 /**
   * Start the socket server
   */
  def startup() {
   
    this.synchronized {
   
		//一个ip地址和连接数量的限额器,每当创建一个新的连接,将调用内部的def inc(address: InetAddress)方法使连接数+1
		//关闭一个连接调用def dec(address: InetAddress)使连接数-1
		//其内部做了限制,每个ip允许创建的最大连接数通过配置读取,count计数器超过这个最大值,会抛出too many connections异常信息
		connectionQuotas = new ConnectionQuotas(maxConnectionsPerIp, maxConnectionsPerIpOverrides)

      val sendBufferSize = config.socketSendBufferBytes
      val recvBufferSize = config.socketReceiveBufferBytes
      val brokerId = config.brokerId

      var processorBeginIndex = 0
      endpoints.values.foreach {
    endpoint =>
        val protocol = endpoint.protocolType
        //processor的数量和配置的线程数相等
        val processorEndIndex = processorBeginIndex + numProcessorThreads
		//创建配置的processor数量实例
        for (i <- processorBeginIndex until processorEndIndex)
          processors(i) = newProcessor(i, connectionQuotas, protocol)
		//根据配置信息创建acceptor
        val acceptor = new Acceptor(endpoint, sendBufferSize, recvBufferSize, brokerId,
          processors.slice(processorBeginIndex, processorEndIndex), connectionQuotas)
          //每一个终端连接分配一个acceptor,用一个map保存起来
        acceptors.put(endpoint, acceptor)
        //启动终端处理器acceptor线程
        Utils.newThread("kafka-socket-acceptor-%s-%d".format(protocol.toString, endpoint.port), acceptor, false).start()
        acceptor.awaitStartup()

        processorBeginIndex = processorEndIndex
      }
    }

先来看看acceptor的实现

 /**
 * Thread that accepts and configures new connections. There is one of these per endpoint.
 * 接收并配置新的连接的线程.每个终端都有各自的acceptor.
 */
private[kafka] class Acceptor(val endPoint: EndPoint,
                              val sendBufferSize: Int,
                              val recvBufferSize: Int,
                              brokerId: Int,
                              processors: Array[Processor],
                              connectionQuotas: ConnectionQuotas) extends AbstractServerThread(connectionQuotas) with KafkaMetricsGroup {
   
	//调用java NIO包下的selector api,开启一个nio selector
  private val nioSelector = NSelector.open()
  //启动socket连接,方法内部有一行代码,配置阻塞方式为false
  //serverChannel.configureBlocking(false)
  val serverChannel = openServerSocket(endPoint.host, endPoint.port)
	
  this.synchronized {
   
    processors.foreach {
    processor =>
    //循环启动processor线程
      Utils.newThread("kafka-network-thread-%d-%s-%d".format(brokerId, endPoint.protocolType.toString, processor.id), processor, false).start()
    }
  }

当处理新的连接请求时,调用accept方法

 /*
   * Accept a new connection
   */
  def accept(key: SelectionKey, processor: Processor)