kafka（十五）：CDH创建topic

一、实现功能

基于CDH平台的kafka创建topic，并且创建消费者和生产者进行测试。

二、环境

1.Centos7.4
2.CDH5.15.2

三、步骤

1.在Kafka配置文件中配置broker在zookeeper中的地址
搜索“zookeeper.chroot”修改为“/kafka02”，如下图。即broker信息保存在zookeeper的/kafka02目录下

备注，这个必须配置，否则后续创建topic的时候，会报错：
CDH-Kafka Error while executing topic command : replication factor: 1 larger than available brokers:0
解决办法参考：
https://blog.csdn.net/u010003835/article/details/88979410

2.重启kafka服务

3.在zookeeper客户端查看

cd /opt/cloudera/parcels/CDH/bin
./zookeeper-client

进入zookeeper客户端后查看broker信息

[zk: localhost:2181(CONNECTED) 0] ls /kafka02/
isr_change_notification    admin                      
consumers                  cluster                    
config                     latest_producer_id_block   
controller                 brokers                    
controller_epoch  

4.创建topic

kafka-topics --zookeeper ip1,ip2,ip3:2181/kafka02 --create --replication-factor 1 --partitions 1 --topic kafkatopic1
结果：
…
19/08/25 13:59:50 INFO admin.AdminUtils$: Topic creation {"version":1,"partitions":{"0":[91]}}
Created topic "kafkatopic1".
19/08/25 13:59:50 INFO zkclient.ZkEventThread: Terminate ZkClient event thread.
19/08/25 13:59:50 INFO zookeeper.ZooKeeper: Session: 0x26cc6cc42130142 closed
19/08/25 13:59:50 INFO zookeeper.ClientCnxn: EventThread shut down for session: 0x26cc6cc42130142

5.创建producer生产者

kafka-console-producer --broker-list ip1:9092,ip2:9092,ip3:9092 --topic kafkatopic1

6.启动consumer消费者

kafka-console-consumer --bootstrap-server ip1:9092,ip2:9092,ip3:9092 --topic kafkatopic1

7.生产者写入数据，消费者可以消费
（1）生产者写入数据

19/08/25 11:31:03 INFO utils.AppInfoParser: Kafka version : 0.11.0-kafka-3.0.0
19/08/25 11:31:03 INFO utils.AppInfoParser: Kafka commitId : unknown
>sdf  dsfs 
>i am new here
>hello kafka

（2）消费者消费

19/08/25 11:31:33 INFO internals.AbstractCoordinator: Successfully joined group console-consumer-2526 with generation 1
19/08/25 11:31:33 INFO internals.ConsumerCoordinator: Setting newly assigned partitions [from_flume01-0] for group console-consumer-2526
sdf  dsfs 
i am new here
hello kafka

8.其他kafka基本测试实例可以参考

https://github.com/RayBreslinwcl/JavaBase/blob/master/src/main/java/com/bigdata/cdh/kafka/OperatorShellUtil.txt

（成功！）

四、参考

1. CDH-Kafka Error while executing topic command : replication factor: 1 larger than available brokers:
https://blog.csdn.net/u010003835/article/details/88979410
2. CDH 中 kafka 的使用注意
https://blog.csdn.net/zimiao552147572/article/details/88557201\