kafka常用命令

1、创建一个topic

创建一个有3分分区，每个分区有3个副本，名字叫kafkatesttopic的topic

[kafka@node1 ~]$ kafka-topics.sh --create --zookeeper 192.168.0.141:2181,192.168.0.142:2181,192.168.0.143:2181/kafkagroup --partitions 3 --replication-factor 3 --topic kafkatesttopic
Created topic kafkatesttopic.

2、列出所有的topic

[kafka@node1 ~]$ kafka-topics.sh --list --zookeeper 192.168.0.141:2181,192.168.0.142:2181,192.168.0.143:2181/kafkagroup
kafkatesttopic

3、查看集群中所有topic的详细信息

[kafka@node1 ~]$ kafka-topics.sh --describe --zookeeper 192.168.0.141:2181,192.168.0.142:2181,192.168.0.143:2181/kafkagroup
Topic:__consumer_offsets        PartitionCount:50       ReplicationFactor:1     Configs:segment.bytes=104857600,cleanup.policy=compact,compression.type=producer
        Topic: __consumer_offsets       Partition: 0    Leader: 143     Replicas: 143   Isr: 143
        Topic: __consumer_offsets       Partition: 1    Leader: 141     Replicas: 141   Isr: 141
        Topic: __consumer_offsets       Partition: 2    Leader: 142     Replicas: 142   Isr: 142
        Topic: __consumer_offsets       Partition: 3    Leader: 143     Replicas: 143   Isr: 143
        Topic: __consumer_offsets       Partition: 4    Leader: 141     Replicas: 141   Isr: 141
        Topic: __consumer_offsets       Partition: 5    Leader: 142     Replicas: 142   Isr: 142
        Topic: __consumer_offsets       Partition: 6    Leader: 143     Replicas: 143   Isr: 143
        Topic: __consumer_offsets       Partition: 7    Leader: 141     Replicas: 141   Isr: 141
        Topic: __consumer_offsets       Partition: 8    Leader: 142     Replicas: 142   Isr: 142
        Topic: __consumer_offsets       Partition: 9    Leader: 143     Replicas: 143   Isr: 143
        Topic: __consumer_offsets       Partition: 10   Leader: 141     Replicas: 141   Isr: 141
        Topic: __consumer_offsets       Partition: 11   Leader: 142     Replicas: 142   Isr: 142
        Topic: __consumer_offsets       Partition: 12   Leader: 143     Replicas: 143   Isr: 143
        Topic: __consumer_offsets       Partition: 13   Leader: 141     Replicas: 141   Isr: 141
        Topic: __consumer_offsets       Partition: 14   Leader: 142     Replicas: 142   Isr: 142
        Topic: __consumer_offsets       Partition: 15   Leader: 143     Replicas: 143   Isr: 143
        Topic: __consumer_offsets       Partition: 16   Leader: 141     Replicas: 141   Isr: 141
        Topic: __consumer_offsets       Partition: 17   Leader: 142     Replicas: 142   Isr: 142
        Topic: __consumer_offsets       Partition: 18   Leader: 143     Replicas: 143   Isr: 143
        Topic: __consumer_offsets       Partition: 19   Leader: 141     Replicas: 141   Isr: 141
        Topic: __consumer_offsets       Partition: 20   Leader: 142     Replicas: 142   Isr: 142
        Topic: __consumer_offsets       Partition: 21   Leader: 143     Replicas: 143   Isr: 143
        Topic: __consumer_offsets       Partition: 22   Leader: 141     Replicas: 141   Isr: 141
        Topic: __consumer_offsets       Partition: 23   Leader: 142     Replicas: 142   Isr: 142
        Topic: __consumer_offsets       Partition: 24   Leader: 143     Replicas: 143   Isr: 143
        Topic: __consumer_offsets       Partition: 25   Leader: 141     Replicas: 141   Isr: 141
        Topic: __consumer_offsets       Partition: 26   Leader: 142     Replicas: 142   Isr: 142
        Topic: __consumer_offsets       Partition: 27   Leader: 143     Replicas: 143   Isr: 143
        Topic: __consumer_offsets       Partition: 28   Leader: 141     Replicas: 141   Isr: 141
        Topic: __consumer_offsets       Partition: 29   Leader: 142     Replicas: 142   Isr: 142
        Topic: __consumer_offsets       Partition: 30   Leader: 143     Replicas: 143   Isr: 143
        Topic: __consumer_offsets       Partition: 31   Leader: 141     Replicas: 141   Isr: 141
        Topic: __consumer_offsets       Partition: 32   Leader: 142     Replicas: 142   Isr: 142
        Topic: __consumer_offsets       Partition: 33   Leader: 143     Replicas: 143   Isr: 143
        Topic: __consumer_offsets       Partition: 34   Leader: 141     Replicas: 141   Isr: 141
        Topic: __consumer_offsets       Partition: 35   Leader: 142     Replicas: 142   Isr: 142
        Topic: __consumer_offsets       Partition: 36   Leader: 143     Replicas: 143   Isr: 143
        Topic: __consumer_offsets       Partition: 37   Leader: 141     Replicas: 141   Isr: 141
        Topic: __consumer_offsets       Partition: 38   Leader: 142     Replicas: 142   Isr: 142
        Topic: __consumer_offsets       Partition: 39   Leader: 143     Replicas: 143   Isr: 143
        Topic: __consumer_offsets       Partition: 40   Leader: 141     Replicas: 141   Isr: 141
        Topic: __consumer_offsets       Partition: 41   Leader: 142     Replicas: 142   Isr: 142
        Topic: __consumer_offsets       Partition: 42   Leader: 143     Replicas: 143   Isr: 143
        Topic: __consumer_offsets       Partition: 43   Leader: 141     Replicas: 141   Isr: 141
        Topic: __consumer_offsets       Partition: 44   Leader: 142     Replicas: 142   Isr: 142
        Topic: __consumer_offsets       Partition: 45   Leader: 143     Replicas: 143   Isr: 143
        Topic: __consumer_offsets       Partition: 46   Leader: 141     Replicas: 141   Isr: 141
        Topic: __consumer_offsets       Partition: 47   Leader: 142     Replicas: 142   Isr: 142
        Topic: __consumer_offsets       Partition: 48   Leader: 143     Replicas: 143   Isr: 143
        Topic: __consumer_offsets       Partition: 49   Leader: 141     Replicas: 141   Isr: 141
Topic:kafkatesttopic    PartitionCount:3        ReplicationFactor:3     Configs:
        Topic: kafkatesttopic   Partition: 0    Leader: 142     Replicas: 142,143,141   Isr: 142,143,141
        Topic: kafkatesttopic   Partition: 1    Leader: 143     Replicas: 143,141,142   Isr: 143,141,142
        Topic: kafkatesttopic   Partition: 2    Leader: 141     Replicas: 141,142,143   Isr: 141,142,143

4、查看某一个topic的所有详细信息

[kafka@node1 ~]$ kafka-topics.sh --describe --zookeeper 192.168.0.141:2181,192.168.0.142:2181,192.168.0.143:2181/kafkagroup --topic kafkatesttopic
Topic:kafkatesttopic    PartitionCount:3        ReplicationFactor:3     Configs:
        Topic: kafkatesttopic   Partition: 0    Leader: 143     Replicas: 143,141,142   Isr: 143,141,142
        Topic: kafkatesttopic   Partition: 1    Leader: 141     Replicas: 141,142,143   Isr: 141,142,143
        Topic: kafkatesttopic   Partition: 2    Leader: 142     Replicas: 142,143,141   Isr: 142,143,141

5、输出一个topic

[kafka@node1 ~]$ kafka-topics.sh --delete --zookeeper 192.168.0.141:2181,192.168.0.142:2181,192.168.0.143:2181/kafkagroup --topic kafkatesttopic
Topic kafkatesttopic is marked for deletion.
Note: This will have no impact if delete.topic.enable is not set to true.

6、使用命令生产/消费消息

1）先创建一个topic

[kafka@node1 ~]$ kafka-topics.sh --create --zookeeper 192.168.0.141:2181,192.168.0.142:2181,192.168.0.143:2181/kafkagroup --partitions 3 --replication-factor 3 --topic kafkatesttopic
Created topic kafkatesttopic.

2）生产消息

[kafka@node1 ~]$ kafka-console-producer.sh --broker-list 192.168.0.141:9092,192.168.0.142:9092,192.168.0.143:9092 --topic kafkatesttopic
>kevin durant  
>carmelo anthony
>kobe bryant

3）消费消息

[kafka@node2 ~]$ kafka-console-consumer.sh --bootstrap-server 192.168.0.141:9092,192.168.0.142:9092 --topic kafkatesttopic --from-beginning
kevin durant
kobe bryant
carmelo anthony

注意：在0.9版本指定的是zookeeper server，0.11开始变成了broker server

7、列出所有消费者群组

[kafka@node2 ~]$ kafka-consumer-groups.sh --bootstrap-server 192.168.0.141:9092,192.168.0.142:9092 --list

注意：在0.9版本指定的是zookeeper server，0.11开始变成了broker server

8、查看某一个topic对应的消息数量

[kafka@node2 ~]$ kafka-run-class.sh  kafka.tools.GetOffsetShell --broker-list 192.168.0.141:9092,192.168.0.142:9092,192.168.0.143:9092 --topic kafkatesttopic --time -1
kafkatesttopic:0:3
kafkatesttopic:1:3
kafkatesttopic:2:3

分别表示在3个分区中的消息的数量，所有分区中的消息数量的和就是消息的总数。

参考文章：
https://www.cnblogs.com/bigben0123/p/10524320.html
https://blog.csdn.net/a2011480169/article/details/82888172