记录下Kafka中bin目录下的基本Command:
# 启动kakfa
/app/gisbdp/kafka/bin/kafka-server-start.sh -daemon /app/gisbdp/kafka/config/server.properties
# 停止kafka
/app/gisbdp/kafka/bin/kafka-server-stop.sh /app/gisbdp/kafka/config/server.properties
# 创建一个新的topic
/app/gisbdp/kafka/bin/kafka-topics.sh --zookeeper node3:2181,node4:2181,node5:2181 --create --topic test_kafka --partitions 3 --replication-factor 3
# 创建生产者
/app/gisbdp/kafka/bin/kafka-console-producer.sh --broker-list node3:9092 --topic test_kafka
# 创建消费者
/app/gisbdp/kafka/bin/kafka-console-consumer.sh --bootstrap-server node3:9092 --topic test_kafka --from-beginning
# 查看所有topic
/app/gisbdp/kafka/bin/kafka-topics.sh --list --zookeeper node3:2181
# 查看指定topic信息
/app/gisbdp/kafka/bin/kafka-topics.sh --zookeeper node3:2181 --describe --topic test_kafka
# 删除指定topic (建议去ZK客户端删除)
/app/gisbdp/kafka/bin/kafka-topics.sh --zookeeper node3:2181,node4:2181,node5:2181 --delete --topic test_kafka
# 查看消费组
/app/gisbdp/kafka/bin/kafka-consumer-groups.sh --bootstrap-server node:9092 --list --new-consumer
# 查看__consumer_offsets的内容( kafka 0.11 版本前)
/app/gisbdp/kafka/bin/kafka-console-consumer.sh --topic __consumer_offsets --bootstrap-server node3:9092 --formatter "kafka.coordinator.GroupMetadataManager\$OffsetsMessageFormatter" --consumer.config /app/gisbdp/kafka/config/consumer.properties --from-beginning
# ( kafka 0.11 版本后)
./bin/kafka-console-consumer.sh --topic __consumer_offsets --bootstrap-server node3:9092 --formatter "kafka.coordinator.group.GroupMetadataManager\$OffsetsMessageFormatter" --consumer.config ./config/consumer.properties --from-beginning
# 看下__consumer_offsets中的内容
[test-consumer-group,__consumer_offsets,45]::[OffsetMetadata[2652,NO_METADATA],CommitTime 1594352464669,ExpirationTime 1594438864669]
[test-consumer-group,__consumer_offsets,1]::[OffsetMetadata[0,NO_METADATA],CommitTime 1594352464669,ExpirationTime 1594438864669]
[test-consumer-group,__consumer_offsets,5]::[OffsetMetadata[0,NO_METADATA],CommitTime 1594352464669,ExpirationTime 1594438864669]
[test-consumer-group,__consumer_offsets,26]::[OffsetMetadata[0,NO_METADATA],CommitTime 1594352464669,ExpirationTime 1594438864669]
[test-consumer-group,__consumer_offsets,29]::[OffsetMetadata[0,NO_METADATA],CommitTime 1594352464669,ExpirationTime 1594438864669]
[test-consumer-group,__consumer_offsets,34]::[OffsetMetadata[0,NO_METADATA],CommitTime 1594352464669,ExpirationTime 1594438864669]
[test-consumer-group,__consumer_offsets,10]::[OffsetMetadata[0,NO_METADATA],CommitTime 1594352464669,ExpirationTime 1594438864669]
[test-consumer-group,__consumer_offsets,32]::[OffsetMetadata[0,NO_METADATA],CommitTime 1594352464669,ExpirationTime 1594438864669]
[test-consumer-group,__consumer_offsets,40]::[OffsetMetadata[0,NO_METADATA],CommitTime 1594352464669,ExpirationTime 1594438864669]
可以看到__consumer_offsets topic的每一日志项的格式都是:[Group, Topic, Partition]::[OffsetMetadata[Offset, Metadata], CommitTime, ExpirationTime]
-- 查看Zookeeper:
get /consumers/ac_online_user/offsets/ac_online_user/0
get /brokers/ids/3