kafka的性能测试

扔这里面记录下

./bin/kafka-topics.sh --create --zookeeper 10.104.109.176:2188,10.104.86.238:2188,10.104.122.21:2188 --replication-factor 2 --partitions 6 --topic kafkatest

./bin/kafka-producer-perf-test.sh  \
--topic kafkatest  \
--throughput 1000000  \
--num-records 1000000  \
--producer-props bootstrap.servers=10.104.109.176:9092,10.104.86.238:9092,10.104.122.21:9092 acks="all" linger.ms=5 \
--record-size 1000


当不设置ack和linger.ms  kafka的写入速度真的挺快的

743495 records sent, 148699.0 records/sec (141.81 MB/sec), 175.7 ms avg latency, 377.0 max latency.
1000000 records sent, 157654.106889 records/sec (150.35 MB/sec), 171.21 ms avg latency, 377.00 ms max latency, 190 ms 50th, 331 ms 95th, 357 ms 99th, 370 ms 99.9th.


设置：acks="all"
243496 records sent, 48699.2 records/sec (46.44 MB/sec), 517.3 ms avg latency, 898.0 max latency.
280770 records sent, 56154.0 records/sec (53.55 MB/sec), 535.3 ms avg latency, 1120.0 max latency.
293340 records sent, 58668.0 records/sec (55.95 MB/sec), 511.0 ms avg latency, 1311.0 max latency.
1000000 records sent, 53262.316911 records/sec (50.79 MB/sec), 537.54 ms avg latency, 1430.00 ms max latency, 361 ms 50th, 1305 ms 95th, 1417 ms 99th, 1427 ms 99.9th.

设置：acks="all"  linger.ms=4
284631 records sent, 56926.2 records/sec (54.29 MB/sec), 436.6 ms avg latency, 667.0 max latency.
311982 records sent, 62396.4 records/sec (59.51 MB/sec), 481.0 ms avg latency, 913.0 max latency.
300600 records sent, 60120.0 records/sec (57.33 MB/sec), 509.2 ms avg latency, 984.0 max latency.
1000000 records sent, 58578.876457 records/sec (55.87 MB/sec), 486.81 ms avg latency, 984.00 ms max latency, 553 ms 50th, 952 ms 95th, 974 ms 99th, 982 ms 99.9th.

设置：acks="all"  linger.ms=5
284366 records sent, 56873.2 records/sec (54.24 MB/sec), 427.2 ms avg latency, 686.0 max latency.
313170 records sent, 62634.0 records/sec (59.73 MB/sec), 485.4 ms avg latency, 842.0 max latency.
309514 records sent, 61902.8 records/sec (59.04 MB/sec), 487.5 ms avg latency, 1003.0 max latency.
1000000 records sent, 59087.686126 records/sec (56.35 MB/sec), 479.60 ms avg latency, 1022.00 ms max latency, 479 ms 50th, 965 ms 95th, 1004 ms 99th, 1018 ms 99.9th.

设置：acks="all" linger.ms=6
281793 records sent, 56358.6 records/sec (53.75 MB/sec), 443.2 ms avg latency, 815.0 max latency.
298170 records sent, 59634.0 records/sec (56.87 MB/sec), 503.2 ms avg latency, 1091.0 max latency.
302494 records sent, 60498.8 records/sec (57.70 MB/sec), 489.3 ms avg latency, 1340.0 max latency.
1000000 records sent, 56233.481415 records/sec (53.63 MB/sec), 503.94 ms avg latency, 1393.00 ms max latency, 363 ms 50th, 1335 ms 95th, 1365 ms 99th, 1390 ms 99.9th.