kafka源码分析（1）KafkaServer

最新推荐文章于 2023-03-01 22:38:05 发布

liangqilang

最新推荐文章于 2023-03-01 22:38:05 发布

阅读量167

点赞数

分类专栏： kafka 文章标签：大数据

本文链接：https://blog.csdn.net/liangqilang/article/details/88331851

版权

kafka 专栏收录该内容

1 篇文章 0 订阅

订阅专栏

版本：kafka_2.10-0.8.1.1

一、kafka-server-start.sh 启动分析

第一步：nohup ./kafka-server-start.sh ../config/server.properties &

第二步：kafka-run-class.sh 调用 kafka.Kafka

第三步：初始化val kafkaServerStartble = new KafkaServerStartable(serverConfig)并startup

第四步：KafkaServerStartable 只是简单封装 KafkaServer，并进行startup、shutdown控制，所以KafkaServer才是kafka启动模块的核心

第五步：启动核心分析-初始化

  private var isShuttingDown = new AtomicBoolean(false)  //初始化关闭标志
  private var shutdownLatch = new CountDownLatch(1)
  private var startupComplete = new AtomicBoolean(false);
  val correlationId: AtomicInteger = new AtomicInteger(0)
  var socketServer: SocketServer = null
  var requestHandlerPool: KafkaRequestHandlerPool = null
  var logManager: LogManager = null
  var kafkaHealthcheck: KafkaHealthcheck = null
  var topicConfigManager: TopicConfigManager = null
  var replicaManager: ReplicaManager = null
  var apis: KafkaApis = null
  var kafkaController: KafkaController = null
   //kafka内部线程池，底层采用ScheduledThreadPoolExecutor
  val kafkaScheduler = new KafkaScheduler(config.backgroundThreads)
  var zkClient: ZkClient = null

第六步：KafkaServer 启动

 info("starting")
    isShuttingDown = new AtomicBoolean(false) //初始化关闭标志
    shutdownLatch = new CountDownLatch(1) //初始化关闭钩子

    /* start scheduler */
    kafkaScheduler.startup() //公共线程池启动
    
    /* setup zookeeper */
    zkClient = initZk()//初始化zk客户端

    /* start log manager */
    logManager = createLogManager(zkClient)
    logManager.startup()

    //初始化服务端 Nio模式
    socketServer = new SocketServer(config.brokerId,
                                    config.hostName,
                                    config.port,
                                    config.numNetworkThreads,
                                    config.queuedMaxRequests,
                                    config.socketSendBufferBytes,
                                    config.socketReceiveBufferBytes,
                                    config.socketRequestMaxBytes)
    socketServer.startup()

    //初始化副本管理器
    replicaManager = new ReplicaManager(config, time, zkClient, kafkaScheduler, logManager, isShuttingDown)
    kafkaController = new KafkaController(config, zkClient)
    
    //初始化API
    /* start processing requests */
    apis = new KafkaApis(socketServer.requestChannel, replicaManager, zkClient, config.brokerId, config, kafkaController)
    requestHandlerPool = new KafkaRequestHandlerPool(config.brokerId, socketServer.requestChannel, apis, config.numIoThreads)
   
    Mx4jLoader.maybeLoad()

    replicaManager.startup()

    kafkaController.startup()
    
     //初始化topic配置管理器
    topicConfigManager = new TopicConfigManager(zkClient, logManager)
    topicConfigManager.startup()
    
    //初始化心跳检测器
    /* tell everyone we are alive */
    kafkaHealthcheck = new KafkaHealthcheck(config.brokerId, config.advertisedHostName, config.advertisedPort, config.zkSessionTimeoutMs, zkClient)
    kafkaHealthcheck.startup()

    //向zk注册本节点
    registerStats()
    startupComplete.set(true);

到此为止kafka已经正常启动，以下是包涵的关键机制，我们一一分析

（1）logManager

（2）SocketServer

（3）ReplicaManager

（4）KafkaController

（5）KafkaApis

（6）TopicConfigManager

（7）KafkaHealthcheck