centos7 系统下 kafka安装
0、前置条件
jdk已安装,zookeeper已安装
1、下载Kafka
官网下载地址: https://kafka.apache.org/downloads
目前最新版本为3.0.0,本文中安装的也是该版本
其中操作命令已经与网上大多数blog中不一样,需要注意
2、配置安装
解压缩到指定路径
tar -zxvf kafka_2.12-3.0.0.tgz -C /usr/local/
修改配置文件
[root@hadoop001 ~]# cat /usr/local/kafka_2.12-3.0.0/config/server.properties | grep -v ^# | grep -v ^$
broker.id=1
listeners=PLAINTEXT://hadoop001:9092
num.network.threads=3
num.io.threads=8
socket.send.buffer.bytes=102400
socket.receive.buffer.bytes=102400
socket.request.max.bytes=104857600
log.dirs=/data/kafka-logs
num.partitions=1
num.recovery.threads.per.data.dir=1
offsets.topic.replication.factor=1
transaction.state.log.replication.factor=1
transaction.state.log.min.isr=1
log.retention.hours=168
log.segment.bytes=1073741824
log.retention.check.interval.ms=300000
zookeeper.connect=hadoop001:2181,hadoop002:2181,hadoop003:2181
zookeeper.connection.timeout.ms=18000
group.initial.rebalance.delay.ms=0
拷贝/usr/local/kafka_2.12-3.0.0 到集群其他节点
修改上述配置文件中的broker.id/listeners
其中broker.id要保证全局唯一
3、启动服务
在各个节点上依次执行如下命令
/usr/local/kafka_2.12-3.0.0/bin/kafka-server-start.sh /usr/local/kafka_2.12-3.0.0/config/server.properties
4、创建topic测试
[root@hadoop001 bin]# ./kafka-topics.sh --create --bootstrap-server hadoop001:9092 --replication-factor 1 --partitions 1 --topic test
Created topic test.
[root@hadoop001 bin]# ./kafka-topics.sh --list --bootstrap-server hadoop001:9092,hadoop002:9092,hadoop003:9092
Hello-Kafka
test
test1