kafka异常之Commit cannot be completed since the group has already rebalanced and assigned the partition

最新推荐文章于 2024-05-31 17:47:49 发布
最新推荐文章于 2024-05-31 17:47:49 发布
阅读量1.7k 收藏
点赞数
文章标签： kafka
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/u014636209/article/details/109802614
版权

1.问题描述

2020-11-19 10:07:14.216  INFO 58352 --- [.1.136_8848-dev] c.a.n.client.config.impl.ClientWorker    : get changedGroupKeys:[]
2020-11-19 10:07:14.218  INFO 58352 --- [ge-manage-group] o.a.k.c.c.internals.AbstractCoordinator  : [Consumer clientId=consumer-3, groupId=charge-manage-group] Group coordinator 172.16.1.168:9092 (id: 2147483646 rack: null) is unavailable or invalid, will attempt rediscovery
2020-11-19 10:07:14.221  INFO 58352 --- [ge-manage-group] o.a.k.c.c.internals.AbstractCoordinator  : [Consumer clientId=consumer-1, groupId=charge-manage-group] Group coordinator 172.16.1.168:9092 (id: 2147483646 rack: null) is unavailable or invalid, will attempt rediscovery
2020-11-19 10:07:14.221  INFO 58352 --- [ge-manage-group] o.a.k.c.c.internals.AbstractCoordinator  : [Consumer clientId=consumer-4, groupId=charge-manage-group] Group coordinator 172.16.1.168:9092 (id: 2147483646 rack: null) is unavailable or invalid, will attempt rediscovery
2020-11-19 10:07:14.236  INFO 58352 --- [ntainer#0-0-C-1] o.a.k.c.c.internals.AbstractCoordinator  : [Consumer clientId=consumer-3, groupId=charge-manage-group] Discovered group coordinator 172.16.1.168:9092 (id: 2147483646 rack: null)
2020-11-19 10:07:14.236  INFO 58352 --- [ntainer#0-0-C-1] o.a.k.c.c.internals.AbstractCoordinator  : [Consumer clientId=consumer-3, groupId=charge-manage-group] Group coordinator 172.16.1.168:9092 (id: 2147483646 rack: null) is unavailable or invalid, will attempt rediscovery
2020-11-19 10:07:14.236  INFO 58352 --- [ntainer#1-0-C-1] o.a.k.c.c.internals.AbstractCoordinator  : [Consumer clientId=consumer-4, groupId=charge-manage-group] Discovered group coordinator 172.16.1.168:9092 (id: 2147483646 rack: null)
2020-11-19 10:07:14.236  INFO 58352 --- [ntainer#1-0-C-1] o.a.k.c.c.internals.AbstractCoordinator  : [Consumer clientId=consumer-4, groupId=charge-manage-group] Group coordinator 172.16.1.168:9092 (id: 2147483646 rack: null) is unavailable or invalid, will attempt rediscovery
2020-11-19 10:07:14.236  INFO 58352 --- [ntainer#2-0-C-1] o.a.k.c.c.internals.AbstractCoordinator  : [Consumer clientId=consumer-1, groupId=charge-manage-group] Discovered group coordinator 172.16.1.168:9092 (id: 2147483646 rack: null)
2020-11-19 10:07:14.237  INFO 58352 --- [ntainer#2-0-C-1] o.a.k.c.c.internals.AbstractCoordinator  : [Consumer clientId=consumer-1, groupId=charge-manage-group] Group coordinator 172.16.1.168:9092 (id: 2147483646 rack: null) is unavailable or invalid, will attempt rediscovery
Creating a new SqlSession
2020-11-19 10:07:14.353  INFO 58352 --- [ntainer#1-0-C-1] o.a.k.c.c.internals.AbstractCoordinator  : [Consumer clientId=consumer-4, groupId=charge-manage-group] Discovered group coordinator 172.16.1.168:9092 (id: 2147483646 rack: null)
2020-11-19 10:07:14.353  INFO 58352 --- [ntainer#0-0-C-1] o.a.k.c.c.internals.AbstractCoordinator  : [Consumer clientId=consumer-3, groupId=charge-manage-group] Discovered group coordinator 172.16.1.168:9092 (id: 2147483646 rack: null)
2020-11-19 10:07:14.354  INFO 58352 --- [ntainer#2-0-C-1] o.a.k.c.c.internals.AbstractCoordinator  : [Consumer clientId=consumer-1, groupId=charge-manage-group] Discovered group coordinator 172.16.1.168:9092 (id: 2147483646 rack: null)
Registering transaction synchronization for SqlSession [org.apache.ibatis.session.defaults.DefaultSqlSession@3e976ae6]
2020-11-19 10:07:14.394  INFO 58352 --- [ntainer#1-0-C-1] o.a.k.c.c.internals.AbstractCoordinator  : [Consumer clientId=consumer-4, groupId=charge-manage-group] Attempt to heartbeat failed for since member id consumer-4-941bbe2e-b367-4c80-b318-cba379a1450d is not valid.
2020-11-19 10:07:14.394 ERROR 58352 --- [ntainer#0-0-C-1] o.a.k.c.c.internals.ConsumerCoordinator  : [Consumer clientId=consumer-3, groupId=charge-manage-group] Offset commit failed on partition sync_product_group-1 at offset 7: The coordinator is not aware of this member.
2020-11-19 10:07:14.394 ERROR 58352 --- [ntainer#1-0-C-1] o.a.k.c.c.internals.ConsumerCoordinator  : [Consumer clientId=consumer-4, groupId=charge-manage-group] Offset commit failed on partition sync_product_group-2 at offset 5: The coordinator is not aware of this member.
2020-11-19 10:07:14.398 ERROR 58352 --- [ntainer#2-0-C-1] o.a.k.c.c.internals.ConsumerCoordinator  : [Consumer clientId=consumer-1, groupId=charge-manage-group] Offset commit failed on partition sync_product_group-0 at offset 7: The coordinator is not aware of this member.
2020-11-19 10:07:14.403 ERROR 58352 --- [ntainer#0-0-C-1] o.a.k.c.c.internals.ConsumerCoordinator  : [Consumer clientId=consumer-3, groupId=charge-manage-group] User provided listener org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer$ListenerConsumerRebalanceListener failed on partition assignment

org.apache.kafka.clients.consumer.CommitFailedException: Commit cannot be completed since the group has already rebalanced and assigned the partitions to another member. This means that the time between subsequent calls to poll() was longer than the configured max.poll.interval.ms, which typically implies that the poll loop is spending too much time message processing. You can address this either by increasing max.poll.interval.ms or by reducing the maximum size of batches returned in poll() with max.poll.records.
	at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator$OffsetCommitResponseHandler.handle(ConsumerCoordinator.java:900) ~[kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator$OffsetCommitResponseHandler.handle(ConsumerCoordinator.java:840) ~[kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.internals.AbstractCoordinator$CoordinatorResponseHandler.onSuccess(AbstractCoordinator.java:978) ~[kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.internals.AbstractCoordinator$CoordinatorResponseHandler.onSuccess(AbstractCoordinator.java:958) ~[kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.internals.RequestFuture$1.onSuccess(RequestFuture.java:204) ~[kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.internals.RequestFuture.fireSuccess(RequestFuture.java:167) ~[kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.internals.RequestFuture.complete(RequestFuture.java:127) ~[kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient$RequestFutureCompletionHandler.fireCompletion(ConsumerNetworkClient.java:578) ~[kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.firePendingCompletedRequests(ConsumerNetworkClient.java:388) ~[kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:294) ~[kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:233) ~[kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:212) ~[kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.commitOffsetsSync(ConsumerCoordinator.java:693) [kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.KafkaConsumer.commitSync(KafkaConsumer.java:1454) [kafka-clients-2.3.1.jar:na]
	at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer$ListenerConsumerRebalanceListener.commitCurrentOffsets(KafkaMessageListenerContainer.java:2323) ~[spring-kafka-2.3.6.RELEASE.jar:2.3.6.RELEASE]
	at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer$ListenerConsumerRebalanceListener.onPartitionsAssigned(KafkaMessageListenerContainer.java:2271) ~[spring-kafka-2.3.6.RELEASE.jar:2.3.6.RELEASE]
	at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.onJoinComplete(ConsumerCoordinator.java:285) [kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.joinGroupIfNeeded(AbstractCoordinator.java:424) [kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureActiveGroup(AbstractCoordinator.java:358) [kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.poll(ConsumerCoordinator.java:353) [kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.KafkaConsumer.updateAssignmentMetadataIfNeeded(KafkaConsumer.java:1251) [kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1216) [kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1201) [kafka-clients-2.3.1.jar:na]
	at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.doPoll(KafkaMessageListenerContainer.java:1034) [spring-kafka-2.3.6.RELEASE.jar:2.3.6.RELEASE]
	at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.pollAndInvoke(KafkaMessageListenerContainer.java:990) [spring-kafka-2.3.6.RELEASE.jar:2.3.6.RELEASE]
	at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.run(KafkaMessageListenerContainer.java:927) [spring-kafka-2.3.6.RELEASE.jar:2.3.6.RELEASE]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_211]
	at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266) [na:1.8.0_211]
	at java.util.concurrent.FutureTask.run(FutureTask.java) [na:1.8.0_211]
	at java.lang.Thread.run(Thread.java:748) [na:1.8.0_211]

2020-11-19 10:07:14.403 ERROR 58352 --- [ntainer#2-0-C-1] o.a.k.c.c.internals.ConsumerCoordinator  : [Consumer clientId=consumer-1, groupId=charge-manage-group] User provided listener org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer$ListenerConsumerRebalanceListener failed on partition assignment

org.apache.kafka.clients.consumer.CommitFailedException: Commit cannot be completed since the group has already rebalanced and assigned the partitions to another member. This means that the time between subsequent calls to poll() was longer than the configured max.poll.interval.ms, which typically implies that the poll loop is spending too much time message processing. You can address this either by increasing max.poll.interval.ms or by reducing the maximum size of batches returned in poll() with max.poll.records.
	at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator$OffsetCommitResponseHandler.handle(ConsumerCoordinator.java:900) ~[kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator$OffsetCommitResponseHandler.handle(ConsumerCoordinator.java:840) ~[kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.internals.AbstractCoordinator$CoordinatorResponseHandler.onSuccess(AbstractCoordinator.java:978) ~[kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.internals.AbstractCoordinator$CoordinatorResponseHandler.onSuccess(AbstractCoordinator.java:958) ~[kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.internals.RequestFuture$1.onSuccess(RequestFuture.java:204) ~[kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.internals.RequestFuture.fireSuccess(RequestFuture.java:167) ~[kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.internals.RequestFuture.complete(RequestFuture.java:127) ~[kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient$RequestFutureCompletionHandler.fireCompletion(ConsumerNetworkClient.java:578) ~[kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.firePendingCompletedRequests(ConsumerNetworkClient.java:388) ~[kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:294) ~[kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:233) ~[kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:212) ~[kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.commitOffsetsSync(ConsumerCoordinator.java:693) [kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.KafkaConsumer.commitSync(KafkaConsumer.java:1454) [kafka-clients-2.3.1.jar:na]
	at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer$ListenerConsumerRebalanceListener.commitCurrentOffsets(KafkaMessageListenerContainer.java:2323) ~[spring-kafka-2.3.6.RELEASE.jar:2.3.6.RELEASE]
	at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer$ListenerConsumerRebalanceListener.onPartitionsAssigned(KafkaMessageListenerContainer.java:2271) ~[spring-kafka-2.3.6.RELEASE.jar:2.3.6.RELEASE]
	at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.onJoinComplete(ConsumerCoordinator.java:285) [kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.joinGroupIfNeeded(AbstractCoordinator.java:424) [kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureActiveGroup(AbstractCoordinator.java:358) [kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.poll(ConsumerCoordinator.java:353) [kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.KafkaConsumer.updateAssignmentMetadataIfNeeded(KafkaConsumer.java:1251) [kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1216) [kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1201) [kafka-clients-2.3.1.jar:na]
	at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.doPoll(KafkaMessageListenerContainer.java:1034) [spring-kafka-2.3.6.RELEASE.jar:2.3.6.RELEASE]
	at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.pollAndInvoke(KafkaMessageListenerContainer.java:990) [spring-kafka-2.3.6.RELEASE.jar:2.3.6.RELEASE]
	at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.run(KafkaMessageListenerContainer.java:927) [spring-kafka-2.3.6.RELEASE.jar:2.3.6.RELEASE]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_211]
	at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266) [na:1.8.0_211]
	at java.util.concurrent.FutureTask.run(FutureTask.java) [na:1.8.0_211]
	at java.lang.Thread.run(Thread.java:748) [na:1.8.0_211]

2020-11-19 10:07:14.403 ERROR 58352 --- [ntainer#1-0-C-1] o.a.k.c.c.internals.ConsumerCoordinator  : [Consumer clientId=consumer-4, groupId=charge-manage-group] User provided listener org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer$ListenerConsumerRebalanceListener failed on partition assignment

org.apache.kafka.clients.consumer.CommitFailedException: Commit cannot be completed since the group has already rebalanced and assigned the partitions to another member. This means that the time between subsequent calls to poll() was longer than the configured max.poll.interval.ms, which typically implies that the poll loop is spending too much time message processing. You can address this either by increasing max.poll.interval.ms or by reducing the maximum size of batches returned in poll() with max.poll.records.
	at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator$OffsetCommitResponseHandler.handle(ConsumerCoordinator.java:900) ~[kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator$OffsetCommitResponseHandler.handle(ConsumerCoordinator.java:840) ~[kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.internals.AbstractCoordinator$CoordinatorResponseHandler.onSuccess(AbstractCoordinator.java:978) ~[kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.internals.AbstractCoordinator$CoordinatorResponseHandler.onSuccess(AbstractCoordinator.java:958) ~[kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.internals.RequestFuture$1.onSuccess(RequestFuture.java:204) ~[kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.internals.RequestFuture.fireSuccess(RequestFuture.java:167) ~[kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.internals.RequestFuture.complete(RequestFuture.java:127) ~[kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient$RequestFutureCompletionHandler.fireCompletion(ConsumerNetworkClient.java:578) ~[kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.firePendingCompletedRequests(ConsumerNetworkClient.java:388) ~[kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:294) ~[kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:233) ~[kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.internals.ConsumerNetworkClient.poll(ConsumerNetworkClient.java:212) ~[kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.commitOffsetsSync(ConsumerCoordinator.java:693) [kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.KafkaConsumer.commitSync(KafkaConsumer.java:1454) [kafka-clients-2.3.1.jar:na]
	at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer$ListenerConsumerRebalanceListener.commitCurrentOffsets(KafkaMessageListenerContainer.java:2323) ~[spring-kafka-2.3.6.RELEASE.jar:2.3.6.RELEASE]
	at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer$ListenerConsumerRebalanceListener.onPartitionsAssigned(KafkaMessageListenerContainer.java:2271) ~[spring-kafka-2.3.6.RELEASE.jar:2.3.6.RELEASE]
	at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.onJoinComplete(ConsumerCoordinator.java:285) [kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.joinGroupIfNeeded(AbstractCoordinator.java:424) [kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.internals.AbstractCoordinator.ensureActiveGroup(AbstractCoordinator.java:358) [kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.internals.ConsumerCoordinator.poll(ConsumerCoordinator.java:353) [kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.KafkaConsumer.updateAssignmentMetadataIfNeeded(KafkaConsumer.java:1251) [kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1216) [kafka-clients-2.3.1.jar:na]
	at org.apache.kafka.clients.consumer.KafkaConsumer.poll(KafkaConsumer.java:1201) [kafka-clients-2.3.1.jar:na]
	at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.doPoll(KafkaMessageListenerContainer.java:1034) [spring-kafka-2.3.6.RELEASE.jar:2.3.6.RELEASE]
	at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.pollAndInvoke(KafkaMessageListenerContainer.java:990) [spring-kafka-2.3.6.RELEASE.jar:2.3.6.RELEASE]
	at org.springframework.kafka.listener.KafkaMessageListenerContainer$ListenerConsumer.run(KafkaMessageListenerContainer.java:927) [spring-kafka-2.3.6.RELEASE.jar:2.3.6.RELEASE]
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511) [na:1.8.0_211]
	at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266) [na:1.8.0_211]
	at java.util.concurrent.FutureTask.run(FutureTask.java) [na:1.8.0_211]
	at java.lang.Thread.run(Thread.java:748) [na:1.8.0_211]

2020-11-19 10:07:14.404  INFO 58352 --- [ntainer#1-0-C-1] o.a.k.c.c.internals.ConsumerCoordinator  : [Consumer clientId=consumer-4, groupId=charge-manage-group] Revoking previously assigned partitions [sync_product_group-2, sync_product-2, sync_rel_service-2]
2020-11-19 10:07:14.404  INFO 58352 --- [ntainer#2-0-C-1] o.a.k.c.c.internals.ConsumerCoordinator  : [Consumer clientId=consumer-1, groupId=charge-manage-group] Revoking previously assigned partitions [sync_product-0, sync_product_group-0, sync_rel_service-0]
2020-11-19 10:07:14.404  INFO 58352 --- [ntainer#0-0-C-1] o.a.k.c.c.internals.ConsumerCoordinator  : [Consumer clientId=consumer-3, groupId=charge-manage-group] Revoking previously assigned partitions [sync_product_group-1, sync_product-1, sync_rel_service-1]
2020-11-19 10:07:14.404  INFO 58352 --- [ntainer#1-0-C-1] o.s.k.l.KafkaMessageListenerContainer    : charge-manage-group: partitions revoked: [sync_product_group-2, sync_product-2, sync_rel_service-2]
2020-11-19 10:07:14.405  INFO 58352 --- [ntainer#1-0-C-1] o.a.k.c.c.internals.AbstractCoordinator  : [Consumer clientId=consumer-4, groupId=charge-manage-group] (Re-)joining group
2020-11-19 10:07:14.404  INFO 58352 --- [ntainer#2-0-C-1] o.s.k.l.KafkaMessageListenerContainer    : charge-manage-group: partitions revoked: [sync_product-0, sync_product_group-0, sync_rel_service-0]
2020-11-19 10:07:14.405  INFO 58352 --- [ntainer#2-0-C-1] o.a.k.c.c.internals.AbstractCoordinator  : [Consumer clientId=consumer-1, groupId=charge-manage-group] (Re-)joining group
2020-11-19 10:07:14.405  INFO 58352 --- [ntainer#2-0-C-1] o.a.k.c.c.internals.AbstractCoordinator  : [Consumer clientId=consumer-1, groupId=charge-manage-group] Attempt to heartbeat failed for since member id  is not valid.
2020-11-19 10:07:14.405  INFO 58352 --- [ntainer#0-0-C-1] o.s.k.l.KafkaMessageListenerContainer    : charge-manage-group: partitions revoked: [sync_product_group-1, sync_product-1, sync_rel_service-1]
2020-11-19 10:07:14.405  INFO 58352 --- [ntainer#0-0-C-1] o.a.k.c.c.internals.AbstractCoordinator  : [Consumer clientId=consumer-3, groupId=charge-manage-group] (Re-)joining group
2020-11-19 10:07:14.405  INFO 58352 --- [ntainer#0-0-C-1] o.a.k.c.c.internals.AbstractCoordinator  : [Consumer clientId=consumer-3, groupId=charge-manage-group] Attempt to heartbeat failed for since member id  is not valid.
2020-11-19 10:07:14.421  INFO 58352 --- [ntainer#2-0-C-1] o.a.k.c.c.internals.AbstractCoordinator  : [Consumer clientId=consumer-1, groupId=charge-manage-group] (Re-)joining group
2020-11-19 10:07:14.422  INFO 58352 --- [ntainer#1-0-C-1] o.a.k.c.c.internals.AbstractCoordinator  : [Consumer clientId=consumer-4, groupId=charge-manage-group] (Re-)joining group
2020-11-19 10:07:14.422  INFO 58352 --- [ntainer#0-0-C-1] o.a.k.c.c.internals.AbstractCoordinator  : [Consumer clientId=consumer-3, groupId=charge-manage-group] (Re-)joining group
2020-11-19 10:07:16.453  INFO 58352 --- [ntainer#3-0-C-1] com.alibaba.druid.pool.DruidDataSource   : {dataSource-2} inited
JDBC Connection [com.mysql.cj.jdbc.ConnectionImpl@12e3753b] will be managed by Spring
2020-11-19 10:07:16.537 DEBUG 58352 --- [ntainer#3-0-C-1] c.b.s.d.c.m.d.m.o.O.selectByExample      : ==>  Preparing: select id, code, name, create_time, update_time, business_type from oms_product_group WHERE ( code = ? ) 
2020-11-19 10:07:16.899 DEBUG 58352 --- [ntainer#3-0-C-1] c.b.s.d.c.m.d.m.o.O.selectByExample      : ==> Parameters: pgcode_gxf(String)
2020-11-19 10:07:17.177 DEBUG 58352 --- [ntainer#3-0-C-1] c.b.s.d.c.m.d.m.o.O.selectByExample      : <==      Total: 0
Releasing transactional SqlSession [org.apache.ibatis.session.defaults.DefaultSqlSession@3e976ae6]
东山富哥
关注
  • 0
    点赞
  • 踩
  • 0
    收藏
    觉得还不错? 一键收藏
  • 打赏
    打赏
  • 1
    评论
flume系列之：Commit cannot be completed since the group has already rebalanced and assigned the partitio
zhengzaifeidelushang的博客
11-06 732
KafkaSource EXCEPTION, {} org.apache.kafka.clients.consumer.CommitFailedException: Commit cannot be completed since the group has already rebalanced and assigned the partitions to another member. This means that the time between subsequent calls to poll()
Commit cannot be completed since the group has already rebalanced and assign
热门推荐
墨痕诉清风的博客
06-08 1万+
如何理解 这里是说提交commit失败, 因为这个组已经重新分配了正常情况下, kafka会有一个配置用于设置一条消息的过期时间, 在规定时间内, 如果消费者提交了消费完成的信息, 那么就可以正常的分配下一条记录给消费者, 并且将当前记录的状态记为"已消费"状态, 对消息队列做一个标识, 避免重复消费kafka中配置的规定返回消息时间, 默认是300s, 也就是5分钟, 但是有一些业务逻辑处理起来比较复杂, 数据量又比较庞大, 那么5分钟是肯定处理不完的, 比如导入一个5G的文件, 然后逐条插入数据库,
org.apache.kafka.clients.consumer.CommitFailedException
最新发布
htydowd的博客
05-31 478
如果消费者A 处理数据 超时，kafka 会将 A 消费者消费的数据分区 重新分配给另一个消费者B 并且 消费者B 会 重新消费 A 没有 提交的数据。调整 max.poll.interval.ms 或者 max.poll.records 能够解决，但是依然避免不了这种问题。3. 解决方案 是 增加 max.poll.interval.ms 设置的时间 或者 减少 max.poll.records 设置的数量。上面两种情况，数据都不会丢，但有可能被重复处理。当然，数据也不会丢，会被其他消费者消费。
Commit cannot be completed since the group has already rebalanced and assigned the partitions to ano
qq_34254090的博客
09-26 574
2021-09-24 14:22:53.040|ERROR|14396|oes_csas_embedded_asset.container-0-C-1|oes-csas-manage||||org.apache.kafka.common.utils.LogContext$KafkaLogger.error-301|[Consumer clientId=consumer-12, groupId=oes-csas-search] Offset commit failed on partition oes_csa
Commit cannot be completed since the group has already rebalanced and assigned the
cyh的博客
02-01 2907
问题场景 我们项目有个订单出库的业务，作为消费者去消费商城项目给我们发的消息。某次突然出现大批订单的出库状态没改的情况，拉日志一看，报了如下异常。 org.apache.kafka.clients.consumer.CommitFailedException: Commit cannot be completed since the group has already rebalanced and assigned the partitions to another member. This means t
kafka Commit cannot be completed since the group has already rebalanced and assigned the partitions
qq_15138049的博客
12-10 711
k8s helm 部署 kafka集群，启动一个producer ，再起一个consumer， 报错如下 root@kafka-0:/# kafka-console-consumer --topic test --bootstrap-server kafka-headless.flink.svc.cluster.local:9092 [2021-12-10 06:20:04,901] ERROR [Consumer clientId=consumer-1, groupId=console-consumer
Kafka the Definitive Guide 2nd Edition
11-22
Kafka the Definitive Guide Kafka 是一个分布式流媒体平台，用于构建实时数据处理和流媒体处理系统。下面是 Kafka 的一些重要知识点： 1. Kafka 概述 Kafka 是一个基于发布/订阅模式的消息队列系统，由 LinkedIn...
Kafka The Definitive Guide Real-Time Data and Stream Processing at Scale epub
10-06
Kafka The Definitive Guide Real-Time Data and Stream Processing at Scale 英文epub 本资源转载自网络，如有侵权，请联系上传者或csdn删除 本资源转载自网络，如有侵权，请联系上传者或csdn删除
Kafka_The Definitive Guide_Real-Time Data and Stream Processing at Scale
11-14
Kafka_The Definitive Guide_Real-Time Data and Stream Processing at Scale, 2017年新书，值得一看，欢迎下载！
Kafka The Definitive Guide
01-27
《Kafka权威指南》是关于实时数据和大规模流处理的权威参考书，由Gwen Shapira、Todd Palino、Rajini Sivaram和Krit Petty等人共同编写。这本书面向的是对Kafka流处理平台感兴趣的软件架构师、开发者以及生产工程师...
kafka自定义partition分发策略实例代码.zip
09-26
kafka版本:kafka_2.11-0.9.0.1 kafka jar包版本:0.9.0.1 kafka集群:192.168.1.101,192.168.1.102,192.168.1.103 partition分发策略主要是自定义Partitioner的实现类,通过根据key和分区数量来实现
kafka报Commit cannot be completed since the group has already rebalanced and assigned the partitions
RanGe的博客
06-23 1万+
Springboot整合kafka 报错 Commit cannot be completed since the group has already rebalanced and assigned the partitions报错信息如何理解产生原因如何解决 报错信息 Commit cannot be completed since the group has already rebalanced and assigned the partitions 如何理解 这里是说提交commit失败, 因为这个
【kafka】kafka报Commit cannot be completed since the group has already rebalanced and assigned the part
DreamSun的博客
10-21 638
Springboot整合kafka 报错 Commit cannot be completed since the group has already rebalanced and assigned the partitions。
Kafka跨集群同步工具——MirrorMaker
Linux平台软件设计与开发
02-16 1万+
MirrorMaker是为解决Kafka跨集群同步、创建镜像集群而存在的；下图展示了其工作原理。该工具消费源集群消息然后将数据重新推送到目标集群。 MirrorMaker使用方式 启动mirror-maker程序需要一个或多个consumer配置文件、一个producer配置文件是必须的其他参数是可选的。 kafka-run-class.sh kafka.tools.MirrorMaker
spring-kafka消费出现异常:Commit cannot be completed since the group has already rebalanced 消费者突然hung住停止消费
huangdi1309的博客
11-02 8729
一天发现线上环境大量报kafka异常：CommitFailedException org.apache.kafka.clients.consumer.CommitFailedException: Commit cannot be completed since the group has already rebalanced and assigned the partitions to another member. This means that the time between subsequent
kafka : CommitFailedException Commit cannot be completed since the group has already rebalanced
九师兄
08-19 2554
背景 读取本地kafka的数据，写入到远程CDH中KUDU表中。 项目引入的kafka版本为 <kafka.version>1.0.1-kafka-3.1.1</kafka.version> <dependency> <groupId>org.apache.kafka</groupId> <artif...
kafka : CommitFailedException already rebalanced and assigned max.poll.records
九师兄
01-16 852
配置参数：session.timeout.ms：是消费者心跳响应的周期，若在这段时间内，没心跳响应，kafka的broker就会移除这个consumer然后rebalance。但是这个不可以，我的kafka 是0.10.0,不能使用kafka-consumer-groups.sh脚本，而使用zk，也是不行的，因为看了，zk中没有存储相关的消费组，有的有，有的没有，因此我采用新建一个消费者使用同一个消费组。实际测试的时候，点击不报错，看着是能修改的，kafka tools也看到修改了。
Commit cannot be completed since the group has already rebalanced and assigned the partitions to another member. This means that the time between subsequent calls to poll() was longer than the configured max.poll.interval.ms, which typically implies that the poll loop is spending too much time message processing. You can address this either by increasing the session timeout or by reducing the maximum size of batches returned in poll() with max.poll.records.问题
06-06
这个问题来自于Kafka消费者，意思是消费者在调用poll()方法时，处理消息的时间太长，导致消费者无法在规定的时间内完成一次poll()操作，从而被认为已经失去了连接。为了解决这个问题，你可以尝试增加会话超时时间或者减少一次poll()方法返回的最大批次记录数（max.poll.records）。这样可以让消费者有更多的时间来处理消息，并避免被认为已经失去连接。
评论 1
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

东山富哥

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付：¥1
获取中
扫码支付

您的余额不足，请更换扫码支付或充值

打赏作者

实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值