k8s helm 部署 kafka集群,启动一个producer ,再起一个consumer, 报错如下
root@kafka-0:/# kafka-console-consumer --topic test --bootstrap-server kafka-headless.flink.svc.cluster.local:9092
[2021-12-10 06:20:04,901] ERROR [Consumer clientId=consumer-1, groupId=console-consumer-46805] Offset commit failed on partition test-1 at offset 3: The request timed out. (org.apache.kafka.clients.consumer.internals.ConsumerCoordinator)
^C^C^C^C[2021-12-10 06:20:32,637] ERROR [Consumer clientId=consumer-1, groupId=console-consumer-46805] Offset commit failed on partition test-1 at offset 3: The coordinator is not aware of this member. (org.apache.kafka.clients.consumer.internals.ConsumerCoordinator)
[2021-12-10 06:20:32,637] WARN [Consumer clientId=consumer-1, groupId=console-consumer-46805] Synchronous auto-commit of offsets {test-1=OffsetAndMetadata{offset=3, metadata=''}, test-0=OffsetAndMetadata{offset=4, metadata=''}, test-2=OffsetAndMetadata{offset=2, metadata=''}} failed: Commit cannot be completed since the group has already rebalanced and assigned the partitions to another member. This means that the time between subsequent calls to poll() was longer than the configured max.poll.interval.ms, which typically implies that the poll loop is spending too much time message processing. You can address this either by increasing the session timeout or by reducing the maximum size of batches returned in poll() with max.poll.records. (org.apache.kafka.clients.consumer.internals.ConsumerCoordinator)
producer
root@kafka-0:/# kafka-console-producer --topic test --broker-list kafka-headless.flink.svc.cluster.local:9092
consumer
root@kafka-0:/# kafka-console-consumer --topic test --bootstrap-server kafka-headless.flink.svc.cluster.local:9092
参考:https://www.cnblogs.com/zhuguangzhe/p/11424460.html
一个consumer在生产消息,另一个consumer在消费它的消息,他们不能在一个groupid 下面,更改其中一个的group id 即可
#########################
1 partitions have leader brokers without a matching listener, including [test-0] (org.apache.kafka.clients.NetworkClient)
一个kafka pod宕了、待其重启后,恢复正常…