kafka windows单机安装测试

初步了解一下一下kafka的基本用法。

win7 64 bit

jdk1.7

1)下载kafka_2.10-0.8.2.1.tgz并解压（整合scala）

2)修改配置文件(可选)

config/zookeeper.properties

  dataDir=/zookeeper

这么写建到d盘根目录

  dataDir=zookeeper

这么写建到bin/windows目录

3)启动zookeeper（窗口1)

cd bin\windows

zookeeper-server-start ..\..\config\zookeeper.properties  

或

zookeeper-server-start ../../config/zookeeper.properties  

4)zookeeper测试 （窗口2)

echo stat | nc 192.168.25.42 2181

D:\fk\tech\kafka\kafka_2.10-0.8.2.1\kafka_2.10-0.8.2.1\bin\windows>zookeeper-she ll.bat 192.168.25.42:2181

注： 这里没有提示符

D:\fk\tech\cloud\zookeeper\zookeeper-3.4.6\bin>zkcli.cmd 192.168.25.42:2181 Connecting to localhost:2181

ls /测试ok

5)启动kafka （窗口3)

kafka-server-start.bat ..\..\config\server.properties

6)创建topic （窗口4)

kafka-topics.bat --create --zookeeper localhost:2181 --replication-factor 1 --partitions 3 --topic mytopic

   
   

    
    

     
     D:\fk\tech\kafka\kafka_2.10-0.8.2.1\kafka_2.10-0.8.2.1\bin\windows>kafka-topics.
    
    
    
    

     
     bat --create --zookeeper localhost:2181 --replication-factor 1 --partitions 3 --
    
    
    
    

     
     topic mytopic
    
    
    
    

     
     Created topic "mytopic".
    
    
   
   

7)查看topic （窗口4)

kafka-topics.bat --describe --zookeeper localhost:2181 --topic  mytopic

   
   

    
    

     
     D:\fk\tech\kafka\kafka_2.10-0.8.2.1\kafka_2.10-0.8.2.1\bin\windows>kafka-topics
    
    
    
    

     
     bat --describe --zookeeper localhost:2181 --topic mytopic
    
    
    
    

     
     Topic:mytopic   PartitionCount:3        ReplicationFactor:1     Configs:
    
    
    
    

     
             Topic: mytopic  Partition: 0    Leader: 0       Replicas: 0     Isr: 0
    
    
    
    

     
             Topic: mytopic  Partition: 1    Leader: 0       Replicas: 0     Isr: 0
    
    
    
    

     
             Topic: mytopic  Partition: 2    Leader: 0       Replicas: 0     Isr: 0
    
    
   
   

8）发布/生产消息窗口 （窗口5)

kafka-console-producer.bat --broker-list localhost:9092 --topic  mytopic

有一个错误提示，但是不影响(kafa的bugKAFKA-1711，KAFKA-1824,预计0.9修复)

   
   

    
    

     
     D:\fk\tech\kafka\kafka_2.10-0.8.2.1\kafka_2.10-0.8.2.1\bin\windows>kafka-consol
    
    
    
    

     
     -producer.bat --broker-list localhost:9092 --topic mytopic
    
    
    
    

     
     [2015-09-30 10:06:11,420] WARN Property topic is not valid (kafka.utils.Verifia
    
    
    
    

     
     leProperties)
    
    
   
   

9）消费消息窗口 （窗口6)

kafka-console-consumer.bat --zookeeper localhost:2181 --from-beginning --topic  mytopic

10）producer窗口输入，可以在consumer窗口输出

    
    

     
     

      
      D:\fk\tech\kafka\kafka_2.10-0.8.2.1\kafka_2.10-0.8.2.1\bin\windows>kafka-console
     
     
     
     

      
      -producer.bat --broker-list localhost:9092 --topic mytopic
     
     
     
     

      
      [2015-09-30 10:06:11,420] WARN Property topic is not valid (kafka.utils.Verifiab
     
     
     
     

      
      leProperties)
     
     
     
     

      
      hello
     
     
     
     

      
      kafka
     
     
     
     

      
      error
     
     
     
     

      
      this is a test msg from producer
     
     
     
     

      
      中文
     
     
    
    

    
    

     
     

      
      D:\fk\tech\kafka\kafka_2.10-0.8.2.1\kafka_2.10-0.8.2.1\bin\windows>kafka-console
     
     
     
     

      
      -consumer.bat --zookeeper localhost:2181 --from-beginning --topic mytopic
     
     
     
     

      
      hello
     
     
     
     

      
      kafka
     
     
     
     

      
      error
     
     
     
     

      
      this is a test msg from producer
     
     
     
     

      
      中文
     
     
    
    

10)znode查看

    
    

     
     

      
      ls /
     
     
     
     

      
      [consumers, config, controller, admin, brokers, zookeeper, controller_epoch
     
     
     
     

      
      ls /consumers
     
     
     
     

      
      [console-consumer-21728]
     
     
     
     

      
      ls /config
     
     
     
     

      
      [topics, changes]
     
     
     
     

      
      ls /controller
     
     
     
     

      
      []
     
     
     
     

      
      ls /admin
     
     
     
     

      
      [delete_topics]
     
     
     
     

      
      ls /brokers
     
     
     
     

      
      [topics, ids]
     
     
     
     

      
      ls /zookeeper
     
     
     
     

      
      [quota]
     
     
     
     

      
      ls /controller_epoch
     
     
     
     

      
      []
     
     
    
    

11)检查offset(窗口4)

 注意：group为consumerid,需要在zk里面查ls /consumer

    
    

     
     

      
      D:\fk\tech\kafka\kafka_2.10-0.8.2.1\kafka_2.10-0.8.2.1\bin\windows>kafka-consume
     
     
     
     

      
      r-offset-checker.bat --topic mytopic  --zookeeper localhost:2181 --broker-info 1
     
     
     
     

      
      27.0.0.1:9092 --group console-consumer-64061
     
     
     
     

      
      Group           Topic                          Pid Offset          logSize
     
     
     
     

      
         Lag             Owner
     
     
     
     

      
      console-consumer-64061 mytopic                        0   4               4
     
     
     
     

      
                0               console-consumer-64061_zynb0411-1443580144073-95285fc3
     
     
     
     

      
      -0
     
     
     
     

      
      console-consumer-64061 mytopic                        1   1               1
     
     
     
     

      
                0               console-consumer-64061_zynb0411-1443580144073-95285fc3
     
     
     
     

      
      -0
     
     
     
     

      
      console-consumer-64061 mytopic                        2   1               1
     
     
     
     

      
                0               console-consumer-64061_zynb0411-1443580144073-95285fc3
     
     
     
     

      
      -0
     
     
     
     

      
      BROKER INFO
     
     
     
     

      
      0 -> zynb0411:9092
     
     
     
     

      
       
     
     
     
     

      
      D:\fk\tech\kafka\kafka_2.10-0.8.2.1\kafka_2.10-0.8.2.1\bin\windows>
     
     
    
    

12)simple-consumer(底层操作) 

指定offset

   
   

    
    

     
     D:\fk\tech\kafka\kafka_2.10-0.8.2.1\kafka_2.10-0.8.2.1\bin\windows>kafka-simple-
    
    
    
    

     
     consumer-shell.bat  --topic mytopic --offset 1 --broker-list 127.0.0.1:9092 --pa
    
    
    
    

     
     rtition 0
    
    
    
    

     
     error
    
    
    
    

     
     this is a test msg from producer
    
    
    
    

     
     中文
    
    
   
   

分别查看不同的partition

    
    

     
     

      
      D:\fk\tech\kafka\kafka_2.10-0.8.2.1\kafka_2.10-0.8.2.1\bin\windows>kafka-simple-
     
     
     
     

      
      consumer-shell.bat  --topic mytopic --offset 0 --broker-list 127.0.0.1:9092 --pa
     
     
     
     

      
      rtition 0
     
     
     
     

      
      kafka
     
     
     
     

      
      error
     
     
     
     

      
      this is a test msg from producer
     
     
     
     

      
      中文
     
     
     
     

      
      终止批处理操作吗(Y/N)? y
     
     
     
     

      
       
     
     
     
     

      
      D:\fk\tech\kafka\kafka_2.10-0.8.2.1\kafka_2.10-0.8.2.1\bin\windows>kafka-simple-
     
     
     
     

      
      consumer-shell.bat  --topic mytopic --offset 0 --broker-list 127.0.0.1:9092 --pa
     
     
     
     

      
      rtition 1
     
     
     
     

      
      go
     
     
     
     

      
      终止批处理操作吗(Y/N)? y
     
     
     
     

      
       
     
     
     
     

      
      D:\fk\tech\kafka\kafka_2.10-0.8.2.1\kafka_2.10-0.8.2.1\bin\windows>kafka-simple-
     
     
     
     

      
      consumer-shell.bat  --topic mytopic --offset 0 --broker-list 127.0.0.1:9092 --pa
     
     
     
     

      
      rtition 2
     
     
     
     

      
      hello
     
     
     
     

      
      终止批处理操作吗(Y/N)? y
     
     
     
     

      
       
     
     
     
     

      
      D:\fk\tech\kafka\kafka_2.10-0.8.2.1\kafka_2.10-0.8.2.1\bin\windows>