Kafka性能测试与分析

最新推荐文章于 2024-07-25 11:32:18 发布

陈冲_河南漯河

最新推荐文章于 2024-07-25 11:32:18 发布

阅读量722

点赞数

分类专栏：大数据

本文链接：https://blog.csdn.net/sod5211314/article/details/86601640

版权

大数据专栏收录该内容

4 篇文章 0 订阅

订阅专栏

写入压力测试kafka-producer-perf-test.sh
--topic             topic名称，本例为test_property
--num-records       总共需要发送的消息数，本例为1000000
--record-size       每个记录的字节数，本例为1000
--throughput        每秒钟发送的记录数，本例为20000
--producer-props    bootstrap.servers=*.*.*.*:9092发送端的配置信息，修改成自己的地址

bin/kafka-producer-perf-test.sh --topic test_property --num-records 1000000 --record-size 1000 --throughput 20000 --producer-props bootstrap.servers=*.*.*.*:9092

./bin/kafka-producer-perf-test.sh --topic test-pati1-rep1 --num-records 1000000 --record-size 1000 --throughput 20000 --producer-props bootstrap.servers=s201:9092,s202:9092,s203:9092

4816 records sent, 952.7 records/sec (0.91 MB/sec), 2497.4 ms avg latency, 3612.0 max latency.

25590 records sent, 4944.9 records/sec (4.72 MB/sec), 6823.4 ms avg latency, 8596.0 max latency.

18900 records sent, 3462.2 records/sec (3.30 MB/sec), 7537.1 ms avg latency, 8601.0 max latency.

10050 records sent, 942.2 records/sec (0.90 MB/sec), 8023.4 ms avg latency, 16464.0 max latency.

18465 records sent, 2679.6 records/sec (2.56 MB/sec), 14808.4 ms avg latency, 18157.0 max latency.

1000000 records sent, 2769.423351 records/sec (2.64 MB/sec), 10100.94 ms avg latency, 73709.00 ms max latency, 6677 ms 50th, 51059 ms 95th, 59888 ms 99th, 61650 ms 99.9th.

./bin/kafka-consumer-perf-test.sh --zookeeper s201:2181,s202:2181,s203:2181 --topic test-pati1-rep1 --fetch-size 1048576 --messages 1000000
start.time, end.time, data.consumed.in.MB, MB.sec, data.consumed.in.nMsg, nMsg.sec
[2019-01-22 05:24:37,444] WARN No broker partitions consumed by consumer thread perf-consumer-66389_s204-1548163476298-31db048d-1 for topic test-pati1-rep1 (kafka.consumer.RangeAssignor)
[2019-01-22 05:24:37,445] WARN No broker partitions consumed by consumer thread perf-consumer-66389_s204-1548163476298-31db048d-2 for topic test-pati1-rep1 (kafka.consumer.RangeAssignor)
[2019-01-22 05:24:37,445] WARN No broker partitions consumed by consumer thread perf-consumer-66389_s204-1548163476298-31db048d-3 for topic test-pati1-rep1 (kafka.consumer.RangeAssignor)
[2019-01-22 05:24:37,446] WARN No broker partitions consumed by consumer thread perf-consumer-66389_s204-1548163476298-31db048d-4 for topic test-pati1-rep1 (kafka.consumer.RangeAssignor)
[2019-01-22 05:24:37,446] WARN No broker partitions consumed by consumer thread perf-consumer-66389_s204-1548163476298-31db048d-5 for topic test-pati1-rep1 (kafka.consumer.RangeAssignor)
[2019-01-22 05:24:37,448] WARN No broker partitions consumed by consumer thread perf-consumer-66389_s204-1548163476298-31db048d-6 for topic test-pati1-rep1 (kafka.consumer.RangeAssignor)
[2019-01-22 05:24:37,449] WARN No broker partitions consumed by consumer thread perf-consumer-66389_s204-1548163476298-31db048d-7 for topic test-pati1-rep1 (kafka.consumer.RangeAssignor)
[2019-01-22 05:24:37,450] WARN No broker partitions consumed by consumer thread perf-consumer-66389_s204-1548163476298-31db048d-8 for topic test-pati1-rep1 (kafka.consumer.RangeAssignor)
[2019-01-22 05:24:37,451] WARN No broker partitions consumed by consumer thread perf-consumer-66389_s204-1548163476298-31db048d-9 for topic test-pati1-rep1 (kafka.consumer.RangeAssignor)
2019-01-22 05:24:38:590, 2019-01-22 05:24:58:303, 872.7407, 44.2723, 915135, 46422.9189