kafka集群

wuxingge

已于 2024-06-20 09:10:09 修改

阅读量192

点赞数

分类专栏： kafka 文章标签： kafka

于 2020-06-22 08:11:17 首次发布

本文链接：https://blog.csdn.net/wuxingge/article/details/106894892

版权

kafka 专栏收录该内容

4 篇文章 0 订阅

订阅专栏

安装jdk1.8

tar xf jdk-8u221-linux-x64.tar.gz -C /opt/
ln -s /opt/jdk1.8.0_221 /opt/jdk

vim /etc/profile

export JAVA_HOME=/opt/jdk
export PATH=$JAVA_HOME/bin:$PATH
export CLASSPATH=$CLASSPATH:$JAVA_HOME/lib:$JAVA_HOME/lib/tools.jar

安装kafka

kafka下载地址
https://kafka.apache.org/downloads

tar xf kafka_2.12-2.2.0.tgz -C /opt/
ln -s /opt/kafka_2.12-2.2.0 /opt/kafka

配置kafka环境变量

vim /etc/profile.d/kafka.sh

export JAVA_HOME=/path/to/jdk
export KAFKA_HOME=/path/to/kafka
export PATH=$KAFKA_HOME/bin:$JAVA_HOME/bin:$PATH
export CLASSPATH=$CLASSPATH:$JAVA_HOME/lib:$JAVA_HOME/lib/tools.jar:$KAFKA_HOME/libs/*

配置(3台都操作)

vim config/server.properties
修改以下配置

broker.id=1
listeners=PLAINTEXT://10.0.0.10:9092
log.dirs=/data/kafka/logs
zookeeper.connect=10.0.0.10:2181,10.0.0.11:2181,10.0.0.12:2181
log.flush.interval.messages=10000
log.flush.interval.ms=1000
delete.topic.enable=true
host.name=es1

broker.id=2
listeners=PLAINTEXT://10.0.0.11:9092
log.dirs=/data/kafka/logs
zookeeper.connect=10.0.0.10:2181,10.0.0.11:2181,10.0.0.12:2181
log.flush.interval.messages=10000
log.flush.interval.ms=1000
delete.topic.enable=true
host.name=es2

broker.id=3
listeners=PLAINTEXT://10.0.0.12:9092
log.dirs=/data/kafka/logs
zookeeper.connect=10.0.0.10:2181,10.0.0.11:2181,10.0.0.12:2181
log.flush.interval.messages=10000
log.flush.interval.ms=1000
delete.topic.enable=true
host.name=es3

mkdir -p /data/kafka/logs

每台启动kafka（要确保zookeeper已启动！！！）

/opt/kafka/bin/kafka-server-start.sh -daemon /opt/kafka/config/server.properties

创建topic

/opt/kafka/bin/kafka-topics.sh --create --zookeeper 10.0.0.11:2181 --replication-factor 3 --partitions 1 --topic test-topic

查看kafka topic列表，使用–list参数

kafka/bin/kafka-topics.sh --list --zookeeper 172.21.0.3:2181

查看创建的topic信息

kafka/bin/kafka-topics.sh --describe --zookeeper 10.0.0.11:2181 --topic topic名称

[root@es2 kafka]# /opt/kafka/bin/kafka-topics.sh --describe --zookeeper 10.0.0.11:2181 --topic test-topic
Topic:test-topic	PartitionCount:1	ReplicationFactor:3	Configs:
	Topic: test-topic	Partition: 0	Leader: 1	Replicas: 1,2,3	Isr: 1,2,3