kafka的shell命令

kafka的 shell 命令

启动集群每个节点的进程

nohup kafka-server-start.sh
/home/hadoop/kafka/kafka_2.11-1.1.0/config/server.properties
1>~/logs/kafka_std.log
2>~/logs/kafka_err.logs &

创建kafka的topic
kafka-topics.sh
–create
–zookeeper hadoop01:2181,hadoop02:2181,hadoop03:2181
–replication-factor 3
–partitions 10
–topic kafka_my01

kafka查看topic的list
kafka-topics.sh
–list
–zookeeper hadoop01:2181,hadoop02:2181,hadoop03:2181

查看某个指定的kafka topic的详细信息：

kafka-topics.sh
–zookeeper hadoop01:2181,hadoop02:2181,hadoop03:2181
–describe
–topic kafka_test01

Topic:kafka_test01 PartitionCount:10 ReplicationFactor:3 Configs:
Topic: kafka_test01 Partition: 0 Leader: 1 Replicas: 1,2,3 Isr: 1,2,3
Topic: kafka_test01 Partition: 1 Leader: 2 Replicas: 2,3,1 Isr: 2,1,3
Topic: kafka_test01 Partition: 2 Leader: 3 Replicas: 3,1,2 Isr: 1,2,3
Topic: kafka_test01 Partition: 3 Leader: 1 Replicas: 1,3,2 Isr: 1,2,3
Topic: kafka_test01 Partition: 4 Leader: 2 Replicas: 2,1,3 Isr: 2,1,3
Topic: kafka_test01 Partition: 5 Leader: 3 Replicas: 3,2,1 Isr: 1,2,3
Topic: kafka_test01 Partition: 6 Leader: 1 Replicas: 1,2,3 Isr: 1,2,3
Topic: kafka_test01 Partition: 7 Leader: 2 Replicas: 2,3,1 Isr: 2,1,3
Topic: kafka_test01 Partition: 8 Leader: 3 Replicas: 3,1,2 Isr: 1,2,3
Topic: kafka_test01 Partition: 9 Leader: 1 Replicas: 1,3,2 Isr: 1,2,3

第一行是对所有分区的一个描述，然后每个分区都会对应一行
Topic：topic 名称
Partition：topic 的分区编号
leader：负责处理消息的读和写，leader 是从所有节点中随机选择的。
replicas：列出了所有的副本节点，不管节点是否在服务中。
isr：正在服务中的节点。

开启生产者模拟生成数据
kafka-console-producer.sh
–broker-list hadoop01:9092,hadoop02:9092,hadoop03:9092
–topic kafka_test01

开启消费者
kafka-console-consumer.sh
–zookeeper hadoop01:2181,hadoop02:2181,hadoop03:2181
–from-beginning
–topic kafka_test01

查看某topic某个分区的偏移量最大值和最小值

kafka-run-class.sh
kafka.tools.GetOffsetShell
–topic kafka_test01
–time -1
–broker-list hadoop01:9092,hadoop02:9092,hadoop03:9092
–partitions 0

增加topic分区数
kafka-topics.sh
–alter
–zookeeper hadoop01:2181,hadoop02:2181,hadoop03:2181
–topic kafka_test01
–partitions 20 \

删除topic
kafka-topics.sh
–delete
–zookeeper hadoop01:2181,hadoop02:2181,hadoop03:2181
–topic kafka_my01