相关报错:
Topic and partition to exceptions: realdeal-0 -> org.apache.kafka.common.errors.RecordTooLargeException
2020-09-18 17:10:12,312 WARN org.apache.kafka.common.network.Selector: [SocketServer brokerId=437] Unexpected error from /172.17.; closing connection
2020-09-18 17:45:00,117 INFO kafka.server.KafkaApis: [KafkaApi-437] Closing connection due to error during produce request with correlation id 4684 from client id producer-520 with ack=0
解决方法:
原因可能是消息太大引起的,将 message.max.bytes 从 900+kb调整为 5Mb
message.max.bytes : kafka 会接收单个消息size的最大限制, 默认为1M左右。如果producer发送比这个大的消息,kafka默认会丢掉。producer可以从callback函数中获得错误码:10。