迷途难返

我要一步一步往上爬,在最高点撑着夜往前飞,让风吹干流过的泪痕,总有一天我要属于我的天...

Kafka Study : Understand the Partitions and replicates

test configurations:
  1. start zookeeper at port 2181
  2. start kafka instance 0 at port 9092 (using default server.properties).
    • kafka_folder] bin/kafka-server-start.sh
  3. start kafka instance 1
    • create new server1.properteis, change the id=1, port=9093 and log directory;
    • bin/kafka-server-start.sh config/server1.properties
  4. start kafka instance 2
    • create new server2.properteis, change the id=2, port=9094 and log directory;
    • bin/kafka-server-start.sh config/server2.properties
  5. create topic: kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 2 --partitions 3 --topic my-replicated-test-topic
  6. check the topic partion and replicates: 
          kafka-topics.sh --describe --topic my-replicated-test-topic --zookeeper localhost:2181
    • Topic:my-replicated-test-topic  PartitionCount:3        ReplicationFactor:2     Configs:
    •        Topic: my-replicated-test-topic Partition: 0    Leader: 0       Replicas: 0,1   Isr: 0,1
    •         Topic: my-replicated-test-topic Partition: 1    Leader: 1       Replicas: 1,2   Isr: 1,2
    •        Topic: my-replicated-test-topic Partition: 2    Leader: 2       Replicas: 2,0   Isr: 2,0

  1. publish message through different kafka instance:
    • kafka-console-producer.sh --broker-list localhost:9092 --topic my-replicated-test-topic
      • 1st message
      • 2nd message
      • 3rd message
    • kafka-console-producer.sh --broker-list localhost:9093 --topic my-replicated-test-topic
      • 4th message
      • 5th message
      • 6th message
    • kafka-console-producer.sh --broker-list localhost:9094 --topic my-replicated-test-topic
      • 7th message
      • 8th message
      • 9th message
  2. check the storage:                   
    kafka instance 0 kafka instance 1 kafka instance 2
    my-replicated-test-topic-replicate-0:
    2nd message, 6th message, 8th message
    my-replicated-test-topic-replicate-0:
    2nd message, 6th message, 8th message
     
      my-replicated-test-topic-replicate-1:
    1st message, 5th message, 7th message
    my-replicated-test-topic-replicate-1:
    1st message, 5th message, 7th message
    my-replicated-test-topic-replicate-2:
    3rd message, 4th message, 9th message
      my-replicated-test-topic-replicate-2:
    3rd message, 4th message, 9th message
     
  3. consumer from beginning from any broker:
  4. kafka-console-consumer.sh --zookeeper localhost:2181 --topic my-replicated-test-topic --from-beginning
  5. 1st message
  6. 2nd message
  7. 5th message
  8. 7th message
  9. 6th message
  10. 8th message
  11. 3rd message
  12. 4th message
  13. 9th message
  14. we can see that: the order from one partition was assured, but from different partitions is not assured
阅读更多
个人分类: 笔记
上一篇Maven Assembly 传递的依赖jars打不进应用程序
想对作者说点什么? 我来说一句

没有更多推荐了,返回首页

关闭
关闭