环境准备:
3台已安装好JDK、Zookeeper的CentOS虚拟机
环境搭建
1) 下载压缩包
https://www.apache.org/dyn/closer.cgi?path=/kafka/2.2.0/kafka_2.12-2.2.0.tgz
2) 解压:
[feng@hadoop129 software]$ ls
kafka_2.11-2.2.0.tgz
[feng@hadoop129 software]$ tar -zxf kafka_2.11-2.2.0.tgz
[feng@hadoop129 software]$ ls
kafka_2.11-2.2.0 kafka_2.11-2.2.0.tgz
[feng@hadoop129 software]$
3) 移动到指定目录:
[feng@hadoop129 software]$ mv kafka_2.11-2.2.0 /opt/module/
4)在/opt/module/kafka_2.11-2.2.0下创建 logs目录:
[feng@hadoop129 kafka_2.11-2.2.0]$ mkdir logs
[feng@hadoop129 kafka_2.11-2.2.0]$ ls
bin config libs LICENSE logs NOTICE site-docs
5) 修改配置文件:
[feng@hadoop129 config]$ vim /opt/module/kafka_2.11-2.2.0/config/server.properties
文件修改、新增内容如下:
#修改
broker.id=129
#新增
#删除topic功能使能
delete.topic.enable=true
#修改 kafka运行日志存放的路径
log.dirs=/opt/module/kafka_2.11-2.2.0/logs
#修改 配置连接Zookeeper集群地址
zookeeper.connect=hadoop129:2181,hadoop130:2181,hadoop131:2181
6)配置环境变量:
[feng@hadoop129 kafka_2.11-2.2.0]$ sudo vim /etc/profile
新增配置,当前生效请执行命令 source /etc/profile:
#KAFAKA_HOME
export KAFAKA_HOME=/opt/module/kafka_2.11-2.2.0
export PATH=$PATH:${KAFAKA_HOME}/bin
7)同步整个文件夹kafaka_2.11-2.2.0到其他两台机子:hadoop130/hadoop131并修改配置文件中的broker.id
hadoop130机子:
broker.id = 130
hadoop131机子:
broker.id = 131
8)先启动zookeeper , 再起动kafka
[feng@hadoop129 kafka_2.11-2.2.0]$ pwd
/opt/module/kafka_2.11-2.2.0
[feng@hadoop129 kafka_2.11-2.2.0]$ bin/kafka-server-start.sh config/server.properties &
[feng@hadoop130 kafka_2.11-2.2.0]$ pwd
/opt/module/kafka_2.11-2.2.0
[feng@hadoop130 kafka_2.11-2.2.0]$ bin/kafka-server-start.sh config/server.properties &
[feng@hadoop131 kafka_2.11-2.2.0]$ pwd
/opt/module/kafka_2.11-2.2.0
[feng@hadoop131 kafka_2.11-2.2.0]$ bin/kafka-server-start.sh config/server.properties &
9)创建并查看topic
[feng@hadoop129 kafka_2.11-2.2.0]$ bin/kafka-topics.sh --create --bootstrap-server hadoop129:9092 --replication-factor 1 --partitions 1 --topic kafka-first
[feng@hadoop129 kafka_2.11-2.2.0]$ bin/kafka-topics.sh --list --bootstrap-server hadoop129:9092
__consumer_offsets
first
first-kafaka-topic
kafka-first
test
10)发送信息
[feng@hadoop129 kafka_2.11-2.2.0]$ bin/kafka-console-producer.sh --broker-list hadoop129:9092 --topic kafka-first
>Hello,Kafka ;-)
>Learning makes you happy
>
11)查看消息
[feng@hadoop129 kafka_2.11-2.2.0]$ bin/kafka-console-consumer.sh --bootstrap-server hadoop129:9092 --topic kafka-first --from-beginning
Hello,Kafka ;-)
Learning makes you happy
12)查看某个Topic的详情
[feng@hadoop129 kafka_2.11-2.2.0]$ bin/kafka-topics.sh --describe --bootstrap-server hadoop129:9092 --topic kafka-first
Topic:kafka-first PartitionCount:1 ReplicationFactor:1 Configs:segment.bytes=1073741824
Topic: kafka-first Partition: 0 Leader: 129 Replicas: 129 Isr: 129
[feng@hadoop129 kafka_2.11-2.2.0]$
13)删除topic
终端执行命令
$ bin/kafka-topics.sh --delete --zookeeper hadoop130:2181 --topic kafka-first
执行前:
执行后,重启kafka、zookeeper:
文章参考了官网的Quick start 和尚学堂的kafka视频教程。