Kafak生产消息测试

1、生产test1-3的消息

[root@localhost ~]# cd kafka_2.12-2.3.0/bin/ #进去kafka的二进制程序目录

[root@localhost bin]# ./kafka-console-producer.sh --broker-list localhost:9092 --topic test-topic #生产者生产消息 (可开一个窗口,模拟生产者)

#生产者生产消息，是往topic里发送消息的，需要指明kafka地址和topic的名字
...
先成为阻塞状态，后续手动输入消息，就是生产消息，生产下面3条消息
...
>test1
>test2
>test3
>...

2、消费消息

[root@localhost ~]# cd kafka_2.12-2.3.0/bin/ #进去kafka的二进制程序目录

[root@localhost bin]# ./kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic test-topic --from-beginning #消费者消费消息(可开多个窗口)

#消费者消费消息，也是从指定topic里取出的，也需要指明kafka地址和topic的名字，加--from-beginning是从头开始收,不加就从当前状态开始收
test1
test2
test3 

 3、查看kafka存放的消息在不同分区：

[root@localhost ~]# ls /root/kafka_2.12-2.3.0/data/test-topic* #消息数据存放在kafka的data目录,有几个分区就分布存储在各个分区
/root/kafka_2.12-2.3.0/data/test-topic-0:
00000000000000000000.index 00000000000000000000.timeindex
00000000000000000000.log leader-epoch-checkpoint

/root/kafka_2.12-2.3.0/data/test-topic-1:
00000000000000000000.index 00000000000000000000.timeindex
00000000000000000000.log leader-epoch-checkpoint

/root/kafka_2.12-2.3.0/data/test-topic-2:
00000000000000000000.index 00000000000000000000.timeindex
00000000000000000000.log leader-epoch-checkpoint

查看具体消息内容
[root@localhost ~]# cat /root/kafka_2.12-2.3.0/data/test-topic-0/00000000000000000000.log
=¹󁃙3·3·ÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
test2[root@localhost ~]# cat /root/kafka_2.12-2.3.0/data/test-topic-1/00000000000000000000.log
=
(83(8ÿÿÿÿÿÿÿÿÿÿÿÿÿÿ
test3[root@localhost ~]# cat /root/kafka_2.12-2.3.0/data/test-topic-2/00000000000000000000.log
=񢁅3񃗳񁀿ÿÿÿÿÿÿÿÿÿÿ
test1=`¥©i89r89rÿÿÿÿÿÿÿÿÿÿÿÿÿÿ 

4、查看kafka中相关元数据信息存放在zookeeper中

[root@localhost ~]# cd /root/zookeeper-3.4.13/bin/ #进入zk的二进制程序目录

[root@localhost bin]# ./zkCli.sh #使用zk客户端命令登录zk

[zk: localhost:2181(CONNECTED) 0] ls / #查看/节点下有哪些数据
[cluster, controller_epoch, controller, brokers, zookeeper, admin, isr_change_notification, consumers, log_dir_event_notification, latest_producer_id_block, config]

[zk: localhost:2181(CONNECTED) 1] ls /brokers #查看brokers下的数据
[ids, topics, seqid]
[zk: localhost:2181(CONNECTED) 2] ls /brokers/ids #查看当前brokers的节点编号
[0]
[zk: localhost:2181(CONNECTED) 3] ls /brokers/topics #查看主题topic的名字
[test-topic, __consumer_offsets]
[zk: localhost:2181(CONNECTED) 4] ls /brokers/topics/test-topic #查看主题下的分区
[partitions]
[zk: localhost:2181(CONNECTED) 5] get /brokers/topics/test-topic #查看主题topic下的相关信息
{"version":1,"partitions":{"2":[0],"1":[0],"0":[0]}} #可看到有3个分区 0 1 2
cZxid = 0x34
ctime = Fri Dec 16 11:59:14 CST 2022
mZxid = 0x34
mtime = Fri Dec 16 11:59:14 CST 2022
pZxid = 0x36
cversion = 1
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 52
numChildren = 1
[zk: localhost:2181(CONNECTED) 6] ls /brokers/topics/test-topic/partitions #查看主题下的partition信息3个分区
[0, 1, 2]