关闭

Kafka Study : Understand the Partitions and replicates

标签: kafkapartitionreplicate
144人阅读 评论(0) 收藏 举报
分类:
test configurations:
  1. start zookeeper at port 2181
  2. start kafka instance 0 at port 9092 (using default server.properties).
    • kafka_folder] bin/kafka-server-start.sh
  3. start kafka instance 1
    • create new server1.properteis, change the id=1, port=9093 and log directory;
    • bin/kafka-server-start.sh config/server1.properties
  4. start kafka instance 2
    • create new server2.properteis, change the id=2, port=9094 and log directory;
    • bin/kafka-server-start.sh config/server2.properties
  5. create topic: kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 2 --partitions 3 --topic my-replicated-test-topic
  6. check the topic partion and replicates: 
          kafka-topics.sh --describe --topic my-replicated-test-topic --zookeeper localhost:2181
    • Topic:my-replicated-test-topic  PartitionCount:3        ReplicationFactor:2     Configs:
    •        Topic: my-replicated-test-topic Partition: 0    Leader: 0       Replicas: 0,1   Isr: 0,1
    •         Topic: my-replicated-test-topic Partition: 1    Leader: 1       Replicas: 1,2   Isr: 1,2
    •        Topic: my-replicated-test-topic Partition: 2    Leader: 2       Replicas: 2,0   Isr: 2,0

  1. publish message through different kafka instance:
    • kafka-console-producer.sh --broker-list localhost:9092 --topic my-replicated-test-topic
      • 1st message
      • 2nd message
      • 3rd message
    • kafka-console-producer.sh --broker-list localhost:9093 --topic my-replicated-test-topic
      • 4th message
      • 5th message
      • 6th message
    • kafka-console-producer.sh --broker-list localhost:9094 --topic my-replicated-test-topic
      • 7th message
      • 8th message
      • 9th message
  2. check the storage:                   
    kafka instance 0 kafka instance 1 kafka instance 2
    my-replicated-test-topic-replicate-0:
    2nd message, 6th message, 8th message
    my-replicated-test-topic-replicate-0:
    2nd message, 6th message, 8th message
     
      my-replicated-test-topic-replicate-1:
    1st message, 5th message, 7th message
    my-replicated-test-topic-replicate-1:
    1st message, 5th message, 7th message
    my-replicated-test-topic-replicate-2:
    3rd message, 4th message, 9th message
      my-replicated-test-topic-replicate-2:
    3rd message, 4th message, 9th message
     
  3. consumer from beginning from any broker:
  4. kafka-console-consumer.sh --zookeeper localhost:2181 --topic my-replicated-test-topic --from-beginning
  5. 1st message
  6. 2nd message
  7. 5th message
  8. 7th message
  9. 6th message
  10. 8th message
  11. 3rd message
  12. 4th message
  13. 9th message
  14. we can see that: the order from one partition was assured, but from different partitions is not assured
0
0

查看评论
* 以上用户言论只代表其个人观点,不代表CSDN网站的观点或立场
    个人资料
    • 访问:5930次
    • 积分:142
    • 等级:
    • 排名:千里之外
    • 原创:7篇
    • 转载:3篇
    • 译文:0篇
    • 评论:0条
    文章分类