Kafka 一些异常报错、注意事项(1)

最新推荐文章于 2023-08-07 00:03:19 发布

MTonj

最新推荐文章于 2023-08-07 00:03:19 发布

阅读量3.4k

点赞数

分类专栏：大数据文章标签： kafka

本文链接：https://blog.csdn.net/MTonj/article/details/124437122

版权

本文记录了Kafka使用中遇到的三个常见问题：消费者poll超时、magic v1不支持record headers异常及分区策略。消费者poll超时可能是由于max_poll_records设置过大导致的；kafka写入异常是因为0.10版本不支持headers，需要升级到0.11以上版本；此外，介绍了Kafka如何通过分区策略在多个broker间分布数据。

摘要由CSDN通过智能技术生成

记录一：This member will leave the group because consumer poll timeout has expired.

[| test_kafka_group] o.a.k.c.c.internals.AbstractCoordinator : [Consumer clientId=consumer-2, groupId=test_kafka_group] This member will leave the group because consumer poll timeout has expired. This means the time between subsequent calls to poll() was longer than the configured max.poll.interval.ms, which typically implies that the poll loop is spending too much time processing messages. You can address this either by increasing max.poll.interval.ms or by reducing the maximum size of batches returned in poll() with max.poll.records.

原因：对kafka的max_poll_records 设置过大，处理不过来。