Kafka Study : Understand the Partitions and replicates

原创 2016年08月31日 09:56:35
test configurations:
  1. start zookeeper at port 2181
  2. start kafka instance 0 at port 9092 (using default server.properties).
    • kafka_folder] bin/kafka-server-start.sh
  3. start kafka instance 1
    • create new server1.properteis, change the id=1, port=9093 and log directory;
    • bin/kafka-server-start.sh config/server1.properties
  4. start kafka instance 2
    • create new server2.properteis, change the id=2, port=9094 and log directory;
    • bin/kafka-server-start.sh config/server2.properties
  5. create topic: kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 2 --partitions 3 --topic my-replicated-test-topic
  6. check the topic partion and replicates: 
          kafka-topics.sh --describe --topic my-replicated-test-topic --zookeeper localhost:2181
    • Topic:my-replicated-test-topic  PartitionCount:3        ReplicationFactor:2     Configs:
    •        Topic: my-replicated-test-topic Partition: 0    Leader: 0       Replicas: 0,1   Isr: 0,1
    •         Topic: my-replicated-test-topic Partition: 1    Leader: 1       Replicas: 1,2   Isr: 1,2
    •        Topic: my-replicated-test-topic Partition: 2    Leader: 2       Replicas: 2,0   Isr: 2,0

  1. publish message through different kafka instance:
    • kafka-console-producer.sh --broker-list localhost:9092 --topic my-replicated-test-topic
      • 1st message
      • 2nd message
      • 3rd message
    • kafka-console-producer.sh --broker-list localhost:9093 --topic my-replicated-test-topic
      • 4th message
      • 5th message
      • 6th message
    • kafka-console-producer.sh --broker-list localhost:9094 --topic my-replicated-test-topic
      • 7th message
      • 8th message
      • 9th message
  2. check the storage:                   
    kafka instance 0 kafka instance 1 kafka instance 2
    my-replicated-test-topic-replicate-0:
    2nd message, 6th message, 8th message
    my-replicated-test-topic-replicate-0:
    2nd message, 6th message, 8th message
     
      my-replicated-test-topic-replicate-1:
    1st message, 5th message, 7th message
    my-replicated-test-topic-replicate-1:
    1st message, 5th message, 7th message
    my-replicated-test-topic-replicate-2:
    3rd message, 4th message, 9th message
      my-replicated-test-topic-replicate-2:
    3rd message, 4th message, 9th message
     
  3. consumer from beginning from any broker:
  4. kafka-console-consumer.sh --zookeeper localhost:2181 --topic my-replicated-test-topic --from-beginning
  5. 1st message
  6. 2nd message
  7. 5th message
  8. 7th message
  9. 6th message
  10. 8th message
  11. 3rd message
  12. 4th message
  13. 9th message
  14. we can see that: the order from one partition was assured, but from different partitions is not assured
版权声明:本文为博主原创文章,未经博主允许不得转载。

相关文章推荐

Apache Kafka笔记(二):Topics,Partitions and Brokers

Topics,Partitions and Replication Factor在消息订阅发布过程中的分析。

11G Concept 第四章翻译 Partitions, Views, and Other Schema Objects(分区,视图,以及其他Schema Objects)

虽然表和索引是最重要、最常用的schema Objects,但是数据库还是很多其他schema objects。其中最常用的都在本章介绍。 本章包含下面几小节: ·Overview of Part...

Understand Wave Format, and Implement a Wave Reader

I found there is no detail discussion for extra data of wave format. Here I post a explicit (for mys...

Understand Linux Shell and Basic Shell Scripting Language Tips-Part I

Picture speak more than words and the below picture says all about the working of Linux.   Und...

Use gitk to understand git – merge and rebase

This is the second part of my Use gitk to understand git post. In my initial overview, I demons...

How to read and understand a scientific paper: a guide for non-scientists

How to read and understand a scientific paper: a guide for non-scientists        网站链接:http://viole...

Item23 Understand std::move and std::forward

这个系列的文章来自于Effective Modern C++的读书笔记,我抽取了其中比较重要的,不容易理解的,平常我们开发过程中也不太在意的一些Item进行分析。 ​ std::move和std:...

Use gitk to understand git – merge and rebase

This is the second part of my Use gitk to understand git post. In my initial overview, I demons...

How to understand and use Milkrun

  • 2011-04-29 16:29
  • 218KB
  • 下载
内容举报
返回顶部
收藏助手
不良信息举报
您举报文章:深度学习:神经网络中的前向传播和反向传播算法推导
举报原因:
原因补充:

(最多只允许输入30个字)