Kafka常用shell脚本命令

1.创建topic

[doda@host166 kafka_2.11-2.4.1]$ ./bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 1 --topic testTopic
Created topic testTopic.

2.查看topic

2.1查看所有topic

[doda@host166 kafka_2.11-2.4.1]$ ./bin/kafka-topics.sh --list --zookeeper localhost:2181
kafkasource
testTopic
zstream_test_topic

2.2查看当前topic

[doda@host166 kafka_2.11-2.4.1]$ ./bin/kafka-topics.sh --zookeeper localhost:2181 --describe --topic testTopic
Topic: testTopic        PartitionCount: 1       ReplicationFactor: 1    Configs: 
        Topic: testTopic        Partition: 0    Leader: 0       Replicas: 0     Isr: 0

3.删除当前topic

[doda@host166 kafka_2.11-2.4.1]$ ./bin/kafka-topics.sh --zookeeper localhost:2181 --delete --topic testTopic
Topic testTopic is marked for deletion.
Note: This will have no impact if delete.topic.enable is not set to true.

4.生产者与消费者使用

4.1生产者启动

[doda@host166 kafka_2.11-2.4.1]$ ./bin/kafka-console-producer.sh --broker-list 172.21.72.166:9092 --topic testTopic
>我是一只小小小鸟，怎么也飞不高
[2022-09-10 17:17:08,629] WARN [Producer clientId=console-producer] Error while fetching metadata with correlation id 3 : {testTopic=LEADER_NOT_AVAILABLE} (org.apache.kafka.clients.NetworkClient)
>

4.2消费者启动

4.2.1从头开始消费

[doda@host166 kafka_2.11-2.4.1]$ ./bin/kafka-console-consumer.sh --bootstrap-server 172.21.72.166:9092 --topic testTopic --from-beginning
我是一只小小小鸟，怎么也飞不高

注:旧版本kafka启动消费参数为zookeeper
./bin/kafka-console-consumer.sh --zookeeper 172.21.72.166:2181 --topic testTopic --from-beginning

4.2.2从最当前最新数据开始消费

[doda@host166 kafka_2.11-2.4.1]$ ./bin/kafka-console-consumer.sh --bootstrap-server 172.21.72.166:9092 --topic testTopic
吹烟袅袅升起，隔江千万里

4.3查看kafak生产者最大位移偏移量

[doda@host166 kafka_2.11-2.4.1]$ ./bin/kafka-run-class.sh kafka.tools.GetOffsetShell --broker-list 172.21.72.166:9092 --topic testTopic --time -1
testTopic:0:8

4.4根据用户组查询各个主题数据消费情况

[doda@host166 kafka_2.11-2.4.1]$ ./bin/kafka-consumer-groups.sh --bootstrap-server 172.21.72.166:9092  --group bigdata  --describe

GROUP           TOPIC           PARTITION  CURRENT-OFFSET  LOG-END-OFFSET  LAG             CONSUMER-ID     HOST            CLIENT-ID
bigdata         user_topic      0          100             200             100             -               -               -
bigdata         test_topic      0          400             400             0               -               -               -