kafka命令行操作

【README】 kafka集群 

-- kafka集群 3台机器 
centos201 192.168.163.201
centos202 192.168.163.202
centos203 192.168.163.203

【1】主题topic 操作命令

序号	命令
1	kafka-topics.sh -- create	新增
2	kafka-topics.sh -- list	查看列表
3	kafka-topics.sh -- delete	删除
4	kafka-topics.sh -- describe	描述

[root@centos201 ~]# kafka-topics.sh --list --zookeeper centos201:2181
[root@centos201 ~]# 
[root@centos201 ~]# kafka-topics.sh --create --zookeeper centos201:2181 --topic first --partitions 2 --replication-factor 2
Created topic "first".
[root@centos201 ~]# 
[root@centos201 ~]# kafka-topics.sh --list --zookeeper centos201:2181                                                      
first

-- log
--partitions 2 2个分区
--replication-factor 2 2个副本

 

 1、创建topic后， topic 即 数据的分类； 其会存储在磁盘，如下：

 

【2】看下 kafak partition 和 replica的数量问题，做个试验 

1） 新建topic，该topic的数据分布在2个分区，每个分区2个副本，如下：

[root@centos201 logs]# kafka-topics.sh --create --zookeeper centos201:2181 --topic sichuan --partitions 2 --replication-factor 2
Created topic "sichuan".

[root@centos201 logs]# kafka-topics.sh --describe --topic sichuan --zookeeper centos201:2181
Topic:sichuan   PartitionCount:2        ReplicationFactor:2     Configs:
        Topic: sichuan  Partition: 0    Leader: 2       Replicas: 2,3   Isr: 2,3
        Topic: sichuan  Partition: 1    Leader: 3       Replicas: 3,1   Isr: 3,1
[root@centos201 logs]# 

2）我们查看 centos201， 202， 203 的 kafka数据， 如下： 

201： sichuan-1

202： sichuan-0 

203：sichuan-0 sichuan-1 

3）在3台机器，新建topic-beijing，设置4个分区，每个分区2个副本，如下：

[root@centos201 logs]# kafka-topics.sh --create --zookeeper centos201:2181 --topic beijing --partitions 4 --replication-factor 2            
Created topic "beijing".
[root@centos201 logs]# 

[root@centos201 logs]# kafka-topics.sh --describe --topic beijing --zookeeper centos201:2181       
Topic:beijing   PartitionCount:4        ReplicationFactor:2     Configs:
        Topic: beijing  Partition: 0    Leader: 1       Replicas: 1,2   Isr: 1,2
        Topic: beijing  Partition: 1    Leader: 2       Replicas: 2,3   Isr: 2,3
        Topic: beijing  Partition: 2    Leader: 3       Replicas: 3,1   Isr: 3,1
        Topic: beijing  Partition: 3    Leader: 1       Replicas: 1,3   Isr: 1,3
[root@centos201 logs]# 

查看topic在磁盘的目录结构 

-- 201
[root@centos201 logs]# ll
total 268
drwxr-xr-x. 2 root root   141 Dec 21 00:07 beijing-0
drwxr-xr-x. 2 root root   141 Dec 21 00:07 beijing-2
drwxr-xr-x. 2 root root   141 Dec 21 00:07 beijing-3

-- 202
[root@centos202 logs]# ll
total 192
drwxr-xr-x. 2 root root   141 Dec 21 00:06 beijing-0
drwxr-xr-x. 2 root root   141 Dec 21 00:06 beijing-1

-- 203 
[root@centos203 logs]# ll
total 188
drwxr-xr-x. 2 root root   141 Dec 20 16:07 beijing-1
drwxr-xr-x. 2 root root   141 Dec 20 16:07 beijing-2
drwxr-xr-x. 2 root root   141 Dec 20 16:07 beijing-3