基于kafka_2.11-0.10.0.1版本:
查看kafka的安装目录
find / -name kafka* -type d
查看redis安装目录:
ps -ef|grep redis
查询所有group信息:
./bin/kafka-topics.sh --list --zookeeper localhost:2181
查询指定topic的详细信息:
./bin/kafka-topics.sh --describe --zookeeper localhost:2181 --topic paymentmgt_download_asyn_deal
示例结果:
Topic:paymentmgt_download_asyn_deal PartitionCount:3 ReplicationFactor:1 Configs:
Topic: paymentmgt_download_asyn_deal Partition: 0 Leader: 0 Replicas: 0 Isr: 0
Topic: paymentmgt_download_asyn_deal Partition: 1 Leader: 0 Replicas: 0 Isr: 0
Topic: paymentmgt_download_asyn_deal Partition: 2 Leader: 0 Replicas: 0 Isr: 0
创建topic:paymentmgt_download_asyn_deal
./bin/kafka-topics.sh --create --zookeeper localhost:2181 --replication-factor 1 --partitions 3 --topic paymentmgt_download_asyn_deal
注:执行后报错:WARNING: Due to limitations in metric names, topics with a period ('.') or underscore ('_') could collide. To avoid issues it is best to use either, but not both.
意思是,由于度量名称的限制,带有句点('.')或下划线的主题可能产生冲突.
topic的名称必须由大小写字母、数字、“.”、“-”、“_”组成,不能为空、不能为“.”、不能为“…”,且长度不能超过249。
topic的命名同样不推荐(虽然可以这样做)使用双下划线“__”开头,因为以双下划线开头的topic一般看作是kafka的内部topic,比如__consumer_offsets和__transaction_state。
查询指定group的topic信息:
./bin/kafka-consumer-groups.sh --zookeeper localhost:2181 --describe --group paymentmgt
查询指定topic的消息内容:
./kafka-console-consumer.sh --zookeeper localhost:2181 --topic paymentmgt_download_asyn_deal --from-beginning
查询匹配指定内容的消息内容:
./bin/kafka-console-consumer.sh --zookeeper 127.0.0.1:2181,127.0.0.1:2182 --topic paymentmgt_download_asyn_deal --from-beginning | grep "1291248658964021250"
消息发送与消息消费示例:
消息发送:
./bin/kafka-console-producer.sh --broker-list 192.168.1.94:9092 --topic paymentmgt_download_asyn_deal
消息消费:
./bin/kafka-console-consumer.sh --zookeeper localhost:2181 --topic paymentmgt_download_asyn_deal
通过查看消费者组信息如下: CURRENT-OFFSET是指当前偏移量, LOG-END-OFFSET是指结束的偏移量, LAG是指落后的偏移量.
GROUP TOPIC PARTITION CURRENT-OFFSET LOG-END-OFFSET LAG OWNER
paymentmgt paymentmgt_download_asyn_deal 0 4 4 0 paymentmgt_192.168.1.94_gfx-0
paymentmgt paymentmgt_download_asyn_deal 1 0 1 1 paymentmgt_192.168.1.94_gfx-0
paymentmgt paymentmgt_download_asyn_deal 2 3 3 0 paymentmgt_192.168.2.33_quw-0