版本信息:
kafka_2.12-3.1.0、apache-zookeeper-3.8.0
报错信息:
[2022-03-16 05:30:57,192] WARN An exception was thrown while closing send thread for session 0x100007216820000. (org.apache.zookeeper.ClientCnxn)
EndOfStreamException: Unable to read additional data from server sessionid 0x100007216820000, likely server has closed socket
at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:77)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:350)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1290)
[2022-03-16 05:30:57,298] INFO Session: 0x100007216820000 closed (org.apache.zookeeper.ZooKeeper)
[2022-03-16 05:30:57,377] INFO EventThread shut down for session: 0x100007216820000 (org.apache.zookeeper.ClientCnxn)
[2022-03-16 05:30:57,380] INFO [ZooKeeperClient Kafka server] Closed. (kafka.zookeeper.ZooKeeperClient)
[2022-03-16 05:30:57,382] ERROR Fatal error during KafkaServer startup. Prepare to shutdown (kafka.server.KafkaServer)
kafka.zookeeper.ZooKeeperClientTimeoutException: Timed out waiting for connection while in state: CONNECTING
at kafka.zookeeper.ZooKeeperClient.$anonfun$waitUntilConnected$3(ZooKeeperClient.scala:254)
at kafka.zookeeper.ZooKeeperClient.waitUntilConnected(ZooKeeperClient.scala:250)
at kafka.zookeeper.ZooKeeperClient.<init>(ZooKeeperClient.scala:108)
at kafka.zk.KafkaZkClient$.apply(KafkaZkClient.scala:1981)
at kafka.zk.KafkaZkClient$.apply(KafkaZkClient.scala:1968)
at kafka.server.KafkaServer.initZkClient(KafkaServer.scala:491)
at kafka.server.KafkaServer.startup(KafkaServer.scala:201)
at kafka.Kafka$.main(Kafka.scala:109)
at kafka.Kafka.main(Kafka.scala)
[2022-03-16 05:30:57,383] INFO shutting down (kafka.server.KafkaServer)
[2022-03-16 05:30:57,470] INFO App info kafka.server for -1 unregistered (org.apache.kafka.common.utils.AppInfoParser)
[2022-03-16 05:30:57,470] INFO shut down completed (kafka.server.KafkaServer)
[2022-03-16 05:30:57,471] ERROR Exiting Kafka. (kafka.Kafka$)
[2022-03-16 05:30:57,471] INFO shutting down (kafka.server.KafkaServer)
第一次尝试解决:
在网上查询资料,说zookeeper的版本需要和kafka安装目录下的libs目录下的(真绕口!)zookeeper的jar包的版本保持一致。重新下载apache-zookeeper-3.6.3-bin.tar.gz,并安装、配置、启动。重新启动kafka,依然报错,报错信息和上面的一致。
第二次尝试解决:
zookeeper的版本不变(v3.6.3)将连接zookeeper的超时时间参数(zookeeper.connection.timeout.ms)的值修改为60000ms,即60秒。重新启动Kafka,进程启动成功。
broker.id=0
listeners=PLAINTEXT://192.168.223.200:9092
advertised.listeners=PLAINTEXT://192.168.223.200:9092
num.network.threads=3
num.io.threads=8
socket.send.buffer.bytes=102400
socket.receive.buffer.bytes=102400
socket.request.max.bytes=104857600
log.dirs=/root/kafka/logs
num.partitions=1
num.recovery.threads.per.data.dir=1
offsets.topic.replication.factor=1
transaction.state.log.replication.factor=1
transaction.state.log.min.isr=1
log.retention.hours=24
log.segment.bytes=1073741824
log.retention.check.interval.ms=300000
zookeeper.connect=192.168.223.200:2181/kafka
zookeeper.connection.timeout.ms=60000
group.initial.rebalance.delay.ms=5
第三次尝试解决:
推倒重来!
将之前的kafka和zookeeper进程停掉,目录全部删除,重新部署、配置和启动,版本就采用kafka_2.12-3.1.0 + apache-zookeeper-3.8.0,采用kafka依旧采用上面的配置,经过测试Kafak进程启动成功。看来问题就是zookeeper.connection.timeout.ms这个默认超时时间18000ms(即18秒不够)导致的,和网上传说的kafka的libs目录下的zookeeper jar包版本与zookeeper版本不一致无关!
Kafka启动脚本 - start.sh(放在/xxx/kafka/bin目录下)
./kafka-server-start.sh -daemon ../config/server.properties
tail -f ../logs/server.log