SpringBoot消费报 groupId Auto-commit of offsets failed

1,错误日志信息如下:

[WARN][2020-08-27 01:22:03 112][org.apache.kafka.clients.consumer.internals.ConsumerCoordinator]-[Auto-commit of offsets {prod_data_sync_trip-0=OffsetAndMetadata{offset=575608778, metadata=''}} failed for group gtdata_sync_ck_prod_data_sync_trip_clickhouse_user_trip: Commit cannot be completed since the group has already rebalanced and assigned the partitions to another member. This means that the time between subsequent calls to poll() was longer than the configured max.poll.interval.ms, which typically implies that the poll loop is spending too much time message processing. You can address this either by increasing the session timeout or by reducing the maximum size of batches returned in poll() with max.poll.records.]
[INFO][2020-08-27 01:22:03 112][org.apache.kafka.clients.consumer.internals.ConsumerCoordinator]-[Revoking previously assigned partitions [prod_data_sync_trip-0] for group gtdata_sync_ck_prod_data_sync_trip_clickhouse_user_trip]
[INFO][2020-08-27 01:22:03 112][org.apache.kafka.clients.consumer.internals.AbstractCoordinator]-[(Re-)joining group gtdata_sync_ck_prod_data_sync_trip_clickhouse_user_trip]
[INFO][2020-08-27 01:22:03 129][org.apache.kafka.clients.consumer.internals.AbstractCoordinator]-[Successfully joined group gtdata_sync_ck_prod_data_sync_trip_clickhouse_user_trip with generation 989]
[INFO][2020-08-27 01:22:03 129][org.apache.kafka.clients.consumer.internals.ConsumerCoordinator]-[Setting newly assigned partitions [prod_data_sync_trip-0] for group gtdata_sync_ck_prod_data_sync_trip_clickhouse_user_trip]

2,通过查看配置,发现 

max.poll.interval.ms 默认配置是300s也就是五分钟,1

而自己的任务是批量插入数据,时间间隔也是五分钟插入一次,如果批量任务有点延迟,就会导致kafka提交offset延迟,从而导致kafka主动端口groupid链接。

 

3,解决方案:

   3.1 加大 max.poll.interval.ms的值把300000 改成 600000

   3.2 批量任务时间间隔缩小,把五分钟执行一次改成三分钟执行一次

  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值