kafka producer无法将数据写入broker

最新推荐文章于 2023-11-03 10:19:02 发布

qq_35725352

最新推荐文章于 2023-11-03 10:19:02 发布

阅读量7.6k

点赞数

分类专栏： kafka 文章标签： kakfa socket

本文链接：https://blog.csdn.net/qq_35725352/article/details/54345506

版权

kafka 专栏收录该内容

1 篇文章 0 订阅

订阅专栏

错误结果：kafka producer无法将数据写入broker
错误报错：
1.log4j DEBUG报错：

2017-01-11 17:09:07,429 [Thread-0] [kafka.producer.async.DefaultEventHandler] [WARN] - Failed to send producer request with correlation id 97 to broker 58 with data for partitions [test,1]
java.io.IOException: 你的主机中的软件中止了一个已建立的连接。
    at sun.nio.ch.SocketDispatcher.writev0(Native Method)
    at sun.nio.ch.SocketDispatcher.writev(Unknown Source)
    at sun.nio.ch.IOUtil.write(Unknown Source)
    at sun.nio.ch.SocketChannelImpl.write(Unknown Source)
    at java.nio.channels.SocketChannel.write(Unknown Source)
    at org.apache.kafka.common.network.ByteBufferSend.writeTo(ByteBufferSend.java:57)
    at kafka.network.RequestOrResponseSend.writeCompletely(RequestOrResponseSend.scala:50)
    at kafka.network.BlockingChannel.send(BlockingChannel.scala:113)
    at kafka.producer.SyncProducer.liftedTree1$1(SyncProducer.scala:80)
    at kafka.producer.SyncProducer.kafka$producer$SyncProducer$$doSend(SyncProducer.scala:79)
	at kafka.producer.SyncProducer$$anonfun$send$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(SyncProducer.scala:110)
	at kafka.producer.SyncProducer$$anonfun$send$1$$anonfun$apply$mcV$sp$1.apply(SyncProducer.scala:110)
	at kafka.producer.SyncProducer$$anonfun$send$1$$anonfun$apply$mcV$sp$1.apply(SyncProducer.scala:110)
	at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33)
	at kafka.producer.SyncProducer$$anonfun$send$1.apply$mcV$sp(SyncProducer.scala:109)
    at kafka.producer.SyncProducer$$anonfun$send$1.apply(SyncProducer.scala:109)
	at kafka.producer.SyncProducer$$anonfun$send$1.apply(SyncProducer.scala:109)
    at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33)
    at kafka.producer.SyncProducer.send(SyncProducer.scala:108)
    at kafka.producer.async.DefaultEventHandler.kafka$producer$async$DefaultEventHandler$$send(DefaultEventHandler.scala:275)
    at kafka.producer.async.DefaultEventHandler$$anonfun$dispatchSerializedData$2.apply(DefaultEventHandler.scala:113)
	at kafka.producer.async.DefaultEventHandler$$anonfun$dispatchSerializedData$2.apply(DefaultEventHandler.scala:105)
    at scala.collection.TraversableLike$WithFilter$$anonfun$foreach$1.apply(TraversableLike.scala:772)
    at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:98)
	at scala.collection.mutable.HashMap$$anonfun$foreach$1.apply(HashMap.scala:98)
    at scala.collection.mutable.HashTable$class.foreachEntry(HashTable.scala:226)
    at scala.collection.mutable.HashMap.foreachEntry(HashMap.scala:39)
    at scala.collection.mutable.HashMap.foreach(HashMap.scala:98)
    at scala.collection.TraversableLike$WithFilter.foreach(TraversableLike.scala:771)
    at kafka.producer.async.DefaultEventHandler.dispatchSerializedData(DefaultEventHandler.scala:105)
    at kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:78)
    at kafka.producer.Producer.send(Producer.scala:78)
    at kafka.javaapi.producer.Producer.send(Producer.scala:35)
    at examples.Producer.run(Producer.java:45)

2.查看/var/log/kafka/server.log报错

2017-01-11 14:23:46,771 ERROR kafka.network.Processor: Closing socket for 192.169.10.193:9092-192.169.10.158:54331 because of error
kafka.network.InvalidRequestException: Error getting request for apiKey: 0 and apiVersion: 2
        at kafka.network.RequestChannel$Request.liftedTree2$1(RequestChannel.scala:93)
        at kafka.network.RequestChannel$Request.<init>(RequestChannel.scala:90)
        at kafka.network.Processor$$anonfun$processCompletedReceives$1.apply(SocketServer.scala:488)
        at kafka.network.Processor$$anonfun$processCompletedReceives$1.apply(SocketServer.scala:483)
        at scala.collection.Iterator$class.foreach(Iterator.scala:742)
        at scala.collection.AbstractIterator.foreach(Iterator.scala:1194)
        at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
        at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
        at kafka.network.Processor.processCompletedReceives(SocketServer.scala:483)
        at kafka.network.Processor.run(SocketServer.scala:413)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: Invalid version for API key 0: 2
        at org.apache.kafka.common.protocol.ProtoUtils.schemaFor(ProtoUtils.java:31)
        at org.apache.kafka.common.protocol.ProtoUtils.requestSchema(ProtoUtils.java:44)
        at org.apache.kafka.common.protocol.ProtoUtils.parseRequest(ProtoUtils.java:60)
        at org.apache.kafka.common.requests.ProduceRequest.parse(ProduceRequest.java:132)
        at org.apache.kafka.common.requests.AbstractRequest.getRequest(AbstractRequest.java:42)
        at kafka.network.RequestChannel$Request.liftedTree2$1(RequestChannel.scala:90)
        ... 10 more