Kafka服务端源码剖析三 -- 启动用于处理请求连接的Processor线程

在创建Acceptor对象的时候，我们初始化了一个processors数组，然后传入进了Acceptor的主构造方法，如下：

endpoints.values.foreach { endpoint =>
        val protocol = endpoint.protocolType
        //todo  numProcessorThreads 网络线程的个数默认为3，至少是1
        val processorEndIndex = processorBeginIndex + numProcessorThreads

        for (i <- processorBeginIndex until processorEndIndex)
          processors(i) = newProcessor(i, connectionQuotas, protocol)

        //TODO 创建Acceptor线程
        //在Acceptor的构造函数中启动了所有的Processor线程
        val acceptor = new Acceptor(endpoint, sendBufferSize, recvBufferSize, brokerId,
          processors.slice(processorBeginIndex, processorEndIndex), connectionQuotas)
        acceptors.put(endpoint, acceptor)
        //TODO 启动acceptor线程
        Utils.newThread("kafka-socket-acceptor-%s-%d".format(protocol.toString, endpoint.port), acceptor, false).start()
        //TODO 等待启动成功
        acceptor.awaitStartup()

        processorBeginIndex = processorEndIndex
      }

接着我们看 Acceptor 的主构造方法中的代码片段：

// 启动所有processor线程
  this.synchronized {
    processors.foreach { processor =>
      Utils
        .newThread("kafka-network-thread-%d-%s-%d".format(brokerId, endPoint.protocolType.toString, processor.id), processor, false)
        .start()
    }
  }

我们看到了Processor线程和Acceptor线程都已经启动了，这些线程之间是如何配合工作的呢？我们再次回到Acceptor线程的run方法里面,看它遍历每个客户端连接的处理逻辑：

//todo 遍历所有的key
            while (iter.hasNext && isRunning) {
              try {
                val key = iter.next
                iter.remove()
                // todo 如果有连接发送过来
                if (key.isAcceptable)
                  //todo 执行此方法，统一对连接进行处理，
                  //todo  对不同的连接采用轮询的方法用不用的Processor来处理。
                  accept(key, processors(currentProcessor))
                else
                  throw new IllegalStateException("Unrecognized key state for acceptor thread.")

                // todo round robin to the next processor thread 
                currentProcessor = (currentProcessor + 1) % processors.length
              } catch {
                case e: Throwable => error("Error while accepting connection", e)
              }
            }

接着点击 accept(key, processors(currentProcessor))如下，
我看见从连接里面获取socketChannel,
最后又调用了processor.accept(socketChannel)

def accept(key: SelectionKey, processor: Processor) {
    val serverSocketChannel = key.channel().asInstanceOf[ServerSocketChannel]
    val socketChannel = serverSocketChannel.accept()
    try {
      connectionQuotas.inc(socketChannel.socket().getInetAddress)
      socketChannel.configureBlocking(false)
      socketChannel.socket().setTcpNoDelay(true)
      socketChannel.socket().setKeepAlive(true)
      if (sendBufferSize != Selectable.USE_DEFAULT_BUFFER_SIZE)
        socketChannel.socket().setSendBufferSize(sendBufferSize)

      debug("Accepted connection from %s on %s and assigned it to processor %d, sendBufferSize [actual|requested]: [%d|%d] recvBufferSize [actual|requested]: [%d|%d]"
            .format(socketChannel.socket.getRemoteSocketAddress, socketChannel.socket.getLocalSocketAddress, processor.id,
                  socketChannel.socket.getSendBufferSize, sendBufferSize,
                  socketChannel.socket.getReceiveBufferSize, recvBufferSize))
      //TODO 每个Processor把Acceptor分配过来的socketChannel添加到自己的队列
      processor.accept(socketChannel)
    } catch {
      case e: TooManyConnectionsException =>
        info("Rejected connection from %s, address already has the configured maximum of %d connections.".format(e.ip, e.count))
        close(socketChannel)
    }
  }

继续跟踪 processor.accept(socketChannel)

 /**
   * Queue up a new connection for reading
   */
  def accept(socketChannel: SocketChannel) {
    newConnections.add(socketChannel)
    wakeup()
  }

最后我们发现 processor线程只是将socketChannel放进了自己的一个队列里面

private val newConnections = new ConcurrentLinkedQueue[SocketChannel]()

所以到这儿我们发现Acceptor把接收到的请求轮训发给不同的Processor，Processor也没有直接对接收到的请求进行处理，而是把存储到了自己队列里面。

总结

我们总结一下Server端目前我们看到的网络部分的情况。

客户端不断的发送请求过来，Acceptor专门来接收各种请求，Acceptor接收到请求以后就把请求轮询的交给不同的Processor，交给多个Processor目的很明确就是要对让每个Processor对自己缓存的连接进行处理，但是目前我们看到的是Processor只是把连接给缓存起来了，下一讲我们剖析一下Processor是如何针对队列里的连接进行处理的？