本文介绍了Kafka中生产者和消费者在吞吐量、低延迟、消息持久性和可用性方面的调优技巧，涉及参数如batch.size、linger.ms、compression.type、acks、buffer.memory等，以及针对不同场景的设置建议。
调优方向：吞吐量
Producer:
• batch.size: increase to 100000–200000 (default 16384)
• linger.ms: increase to 10–100 (default 0)
• compression.type=lz4 (default none, i.e., no compression)
• acks=1 (default 1)
• buffer.memory: increase if there are a lot of partitions (default 33554432)
Consumer:
• fetch.min.bytes: increase to ~100000 (default 1)
调优方向：低延迟
Producer:
• linger.ms=0 (default 0)
• compression.type=none (default none, i.e., no compression)
• acks=1 (default 1)
Consumer:
• fetch.min.bytes=1 (default 1)
调优方向：持久性（消息不丢失）
replication.factor=3
• acks=all (default 1)
• enable.idempotence=true (default false), to prevent duplicate messages and out-of-order
messages
• max.in.flight.requests.per.connection=1 (default 5), to prevent out of order messages
when not using an idempotent producer
Consumer:
• enable.auto.commit=false (default true)
• isolation.level=read_committed (when using EOS transactions)
调优方向：可用性
Consumer:
• session.timeout.ms: increase (default 10000)
max.poll.records减少
1413
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
