kafka命令行操作

饭吃三分饱

于 2024-03-11 20:14:35 发布

阅读量181

点赞数 1

文章标签： kafka linq 分布式

本文链接：https://blog.csdn.net/Wqr_18390921824/article/details/136405634

版权

1 查看kafka版本

# 集群中kafka版本为2.0.1

# 查询kafka版本
# ./kafka-run-class.sh --version
2.0.0 (Commit:3402a8361b734732)

2 topic

<---topic--->
#查看topics
# 版本原因无法使用--bootstrap-server，这里使用--zookeeper
./kafka-topics.sh --zookeeper 192.168.8.58:2181 --list


# 创建topic
# ./kafka-topics.sh --zookeeper 192.168.8.58:2181 --topic first --create --partitions 1 --replication-factor 3
Created topic "first".

# 查看topic 详细信息
# ./kafka-topics.sh --zookeeper 192.168.8.58:2181 --topic first --describe
Topic:first     PartitionCount:1        ReplicationFactor:3     Configs:
        Topic: first    Partition: 0    Leader: 0       Replicas: 0,1,2 Isr: 0,1,2
#PartitionCount：分区数
ReplicationFactor：副本数
Partition：分区为1
Leader：副本的Leader为副本0，生产者和消费者针对副本0进行操作
Replicas: 副本设置3个，所以这里是副本0,1,2

#修改topic，注意partitions只能增加不能减少，不能通过命令行的形式修改副本--replication-factor
#  ./kafka-topics.sh --zookeeper 192.168.8.58:2181 --topic first --alter --partitions 3
WARNING: If partitions are increased for a topic that has a key, the partition logic or ordering of the messages will be affected
Adding partitions succeeded!

#修改topic，再次查看topic详细信息
# ./kafka-topics.sh --zookeeper 172.25.112.1:2181 --topic first --describe
Topic:first     PartitionCount:3        ReplicationFactor:3     Configs:
        Topic: first    Partition: 0    Leader: 0       Replicas: 0,1,2 Isr: 0,1,2
        Topic: first    Partition: 1    Leader: 1       Replicas: 1,2,0 Isr: 1,2,0
        Topic: first    Partition: 2    Leader: 2       Replicas: 2,0,1 Isr: 2,0,1

3 kafka-console-producer


<---kafka-console-producer--->
# 无指定消费组id
./kafka-console-producer.sh --broker-list 192.168.8.58:9092 --topic first
>

# 指定消费组id
./kafka-console-producer.sh --broker-list 192.168.8.58:9092 --topic first --group wuqirui

4 kafka-console-consumer


<---kafka-console-consumer--->
# 当前消息
./kafka-console-consumer.sh --bootstrap-server 192.168.8.58:31092 --topic first

# 历史消息
./kafka-console-consumer.sh --bootstrap-server 192.168.8.58:9092 --topic first --from-beginning

# 查看消费组详细信息
./kafka-console-consumer.sh --bootstrap-server 192.168.8.58:9092 --describe --group first

5. 查看kafka index和log信息

# ./kafka-run-class.sh kafka.tools.DumpLogSegments --files /tmp/kafka-logs/first-0/00000000000000000000.log
Dumping /tmp/kafka-logs/first-0/00000000000000000000.log
Starting offset: 0
baseOffset: 0 lastOffset: 4 count: 5 baseSequence: -1 lastSequence: -1 producerId: -1 producerEpoch: -1 partitionLeaderEpoch: 0 isTransactional: false position: 0 CreateTime: 1709443733976 isvalid: true size: 136 magic: 2 compresscodec: NONE crc: 3174946126
baseOffset: 5 lastOffset: 9 count: 5 baseSequence: 0 lastSequence: 4 producerId: 6000 producerEpoch: 0 partitionLeaderEpoch: 0 isTransactional: true position: 136 CreateTime: 1709448807614 isvalid: true size: 161 magic: 2 compresscodec: NONE crc: 3671548032


# ./kafka-run-class.sh kafka.tools.DumpLogSegments --files /tmp/kafka-logs/first-0/00000000000000000000.index
Dumping /tmp/kafka-logs/first-0/00000000000000000000.index
offset: 0 position: 0