一、报错信息
Kafka访问多次出现超时现象、生产者无法正常发送消息,具体异常如下所示:
org.apache.kafka.common.errors.TimeoutException: Expiring 2 record(s) for coresys_cms_output_contentchange-1: 10692 ms has passed since batch creation plus linger time
2022-03-09 10:23:52.219 [messageListenerChannelChangeContainer-kafka-consumer-1] INFO o.s.k.l.KafkaMessageListenerContainer - partitions assigned:[coresys_cms_output_channelchange-3, coresys_cms_output_channelchange-4, coresys_cms_output_channelchange-1, coresys_cms_output_channelchange-2, coresys_cms_output_channelchange-0]
2022-03-09 10:23:52.220 [kafka-producer-network-thread | producer-1] ERROR o.a.k.c.p.internals.ProducerBatch - Error executing user-provided callback on message for topic-partition 'coresys_cms_output_contentchange-1'
java.lang.RuntimeException: Failed to send kafka.
at com.viewstar.cms.api.cmsChangeNotify.service.KafkaProducerAdapter$SendCallback.onCompletion(KafkaProducerAdapter.java:88)
at org.apache.kafka.clients.producer.KafkaProducer$InterceptorCallback.onCompletion(KafkaProducer.java:1199)
at org.apache.kafka.clients.producer.internals.ProducerBatch.completeFutureAndFireCallbacks(ProducerBatch.java:204)
at org.apache.kafka.clients.producer.internals.ProducerBatch.done(ProducerBatch.java:187)
at org.apache.kafka.clients.producer.internals.Sender.failBatch(Sender.java:627)
at org.apache.kafka.clients.producer.internals.Sender.sendProducerData(Sender.java:287)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:238)
at org.apache.kafka.clients.producer.internals.Sender.run(Sender.java:163)
二、解决方案
1、从错误信息可以看出是超时引起的问题,登录服务器试了试查询Topic情况
./kafka-topics.sh --list --zookeeper x.x.x.x:2181
2、执行第一步命令后发现信息返回需要40s以上,甚至出现zookeeper连接超时,往下排查防火墙情况
service iptables status
结果发现集群中有部分防火墙是开启状态
3、关闭防火墙,服务恢复。