test configurations:
- start zookeeper at port 2181
- start kafka instance 0 at port 9092 (using default server.properties).
- kafka_folder] bin/kafka-server-start.sh
- start kafka instance 1
- create new server1.properteis, change the id=1, port=9093 and log directory;
- bin/kafka-server-start.sh config/server1.properties
- start kafka instance 2
- create new server2.properteis, change the id=2, port=9094 and log directory;
- bin/kafka-server-start.sh config/server2.properties
- create topic: kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 2 --partitions 3 --topic my-replicated-test-topic
- check the topic partion and replicates:
kafka-topics.sh --describe --topic my-replicated-test-topic --zookeeper localhost:2181
-
- Topic:my-replicated-test-topic PartitionCount:3 ReplicationFactor:2 Configs:
- Topic: my-replicated-test-topic Partition: 0 Leader: 0 Replicas: 0,1 Isr: 0,1
- Topic: my-replicated-test-topic Partition: 1 Leader: 1 Replicas: 1,2 Isr: 1,2
- Topic: my-replicated-test-topic Partition: 2 Leader: 2 Replicas: 2,0 Isr: 2,0
- publish message through different kafka instance:
- kafka-console-producer.sh --broker-list localhost:9092 --topic my-replicated-test-topic
-
- 1st message
- 2nd message
- 3rd message
- kafka-console-producer.sh --broker-list localhost:9093 --topic my-replicated-test-topic
-
- 4th message
- 5th message
- 6th message
- kafka-console-producer.sh --broker-list localhost:9094 --topic my-replicated-test-topic
-
- 7th message
- 8th message
- 9th message
- check the storage:
kafka instance 0 kafka instance 1 kafka instance 2 my-replicated-test-topic-replicate-0:
2nd message, 6th message, 8th messagemy-replicated-test-topic-replicate-0:
2nd message, 6th message, 8th messagemy-replicated-test-topic-replicate-1:
1st message, 5th message, 7th messagemy-replicated-test-topic-replicate-1:
1st message, 5th message, 7th messagemy-replicated-test-topic-replicate-2:
3rd message, 4th message, 9th messagemy-replicated-test-topic-replicate-2:
3rd message, 4th message, 9th message - consumer from beginning from any broker:
- kafka-console-consumer.sh --zookeeper localhost:2181 --topic my-replicated-test-topic --from-beginning
- 1st message
- 2nd message
- 5th message
- 7th message
- 6th message
- 8th message
- 3rd message
- 4th message
- 9th message
- we can see that: the order from one partition was assured, but from different partitions is not assured