kafka命令行操作练习

1）查看当前服务器中的所有topic

[root@hadoop101 kafka]# bin/kafka-topics.sh --zookeeper hadoop101:2181 --list
first
liang

2）创建topic

[root@hadoop101 kafka]# bin/kafka-topics.sh --zookeeper hadoop101:2181 --create --replication-factor 3 --partitions 1 --topic test
Created topic "test".

选项说明：
–topic 定义topic名
–replication-factor 定义副本数（副本数<=brokers）
–partitions 定义分区数

3）删除topic

[root@hadoop101 kafka]# bin/kafka-topics.sh --zookeeper hadoop101:2181 --delete--topic first

4）发送消息

[root@hadoop101 kafka]# bin/kafka-console-producer.sh --broker-list hadoop101:9092 --topic test
>hello world
>hello kafka 

5）消费消息（在另外一台机器上开启）

[root@hadoop102*** kafka]# bin/kafka-console-consumer.sh --topic test --zookeeper hadoop101:2181
Using the ConsoleConsumer with old consumer is deprecated and will be removed in a future major release. Consider using the new consumer by passing [bootstrap-server] instead of [zookeeper].
hello word
hello kafka

在第三台机器上启动消费消息重头接受数据

[root@hadoop103 kafka]$ bin/kafka-console-consumer.sh --topic test --zookeeper hadoop101:2181 --from-beginning
Using the ConsoleConsumer with old consumer is deprecated and will be removed in a future major release. Consider using the new consumer by passing [bootstrap-server] instead of [zookeeper].
hello kafka
hello word
hello you

采用bootstrap-server 消费

[root@hadoop102 kafka]# bin/kafka-console-consumer.sh --topic test --bootstrap-server hadoop101:9092
mr.liang

6）查看某个Topic的详情

[root@hadoop101 kafka]# bin/kafka-topics.sh --zookeeper hadoop101:2181 --describe --topic liang
Topic:liang	PartitionCount:3	ReplicationFactor:2	Configs:
	Topic: liang	Partition: 0	Leader: 1	Replicas: 1,0	Isr: 0,1
	Topic: liang	Partition: 1	Leader: 2	Replicas: 2,1	Isr: 1,2
	Topic: liang	Partition: 2	Leader: 0	Replicas: 0,2	Isr: 0,2

7）修改分区数

[root@hadoop101 kafka]#$bin/kafka-topics.sh --zookeeper hadoop101:2181 --alter --topic test --partitions 3

二、数据与日志分离
删除logs

[lzl@hadoop101 kafka]$ sudo rm -rf logs

还要删除zookeeper 的消息
启动zkCli.sh 查看都有哪些消息

[zk: localhost:2181(CONNECTED) 1] ls /
[cluster, controller_epoch, brokers, zookeeper, spark, sanguo, admin, isr_change_notification, consumers, latest_producer_id_block, config]

停止zookeeper后删除zkData中的version-2/

[lzl@hadoop101 zkData]$ rm -rf version-2/

重新启动zookeeper
将kafka中的日志路径改为如下（每一台机器都要改）：

在kafka目录下mkdir data（每台机器都要）
重启kafka
测试：

//创建新主题test
[lzl@hadoop101 kafka]$ bin/kafka-topics.sh --zookeeper hadoop101:2181 --topic test --partitions 2 --replication-factor 2 --create
Created topic "test".