一 、Flume 启动异常:
2017-11-24 17:24:39,115 (kafka-producer-network-thread | producer-1) [DEBUG - org.apache.kafka.common.network.Selector.poll(Selector.java:307)] Connection with /xxxx disconnected
java.io.EOFException
at org.apache.kafka.common.network.NetworkReceive.readFromReadableChannel(NetworkReceive.java:83)
at org.apache.kafka.common.network.NetworkReceive.readFrom(NetworkReceive.java:71)
at org.apache.kafka.common.network.KafkaChannel.receive(KafkaChannel.java:153)
at org.apache.kafka.common.network.KafkaChannel.read(KafkaChannel.java:134)
at org.apache.kafka.common.network.Selector.poll(Selector.java:286)
at org.apache.kafka.clients.NetworkClient.poll(NetworkClient.java:256)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:216)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:128)
at java.lang.Thread.run(Thread.java:745)
因为当时还迁移了zookeeper节点和kafka节点。以为是zookeeper或者kafka的问题。查看zookeeper发现如下异常:
WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn@362] - Exception causing close of session 0x0 due to java.io.IOException: Unreasonable length = 1467246
意思是报文长度过长,不合理。最后发现是批量修改flume节点kafka配置出了问题。kafka.bootstrap.servers的端口配置成了zookeeper端口2181。应该是kafka端口9092。
总之这个问题,是Flume kafka配置错误,导致的连接异常。