Kafka 单节点多Broker部署及使用

配置Kafka

拷贝server.properties三份

cd $KAFKA_HOME/config
cp server.properties server-1.properties
 
cp server.properties server-2.properties
 
cp server.properties server-3.properties

修改server-1.properties文件

# broker的全局唯一编号，不能重复
broker.id=1
# 监听
listeners=PLAINTEXT://:9093
# 日志目录
log.dirs=/home/hadoop/kafka-logs-1

修改server-2.properties文件

# broker的全局唯一编号，不能重复
broker.id=2
# 监听
listeners=PLAINTEXT://:9094
# 日志目录
log.dirs=/home/hadoop/kafka-logs-2

修改server-3.properties文件

# broker的全局唯一编号，不能重复
broker.id=3
# 监听
listeners=PLAINTEXT://:9095
# 日志目录
log.dirs=/home/hadoop/kafka-logs-3

启动Zookeeper

[hadoop@hadoop001]$zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /opt/zookeeper/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
[hadoop@hadoop001 bin]$ ./zkServer.sh  status
JMX enabled by default
Using config: /home/hadoop/app/zookeeper/bin/../conf/zoo.cfg
Mode: standalone
[hadoop@hadoop001 bin]$ 

启动Kafka（分别启动server1、2、3）

$ kafka-server-start.sh $KAFKA_HOME/config/server-1.properties
$ kafka-server-start.sh $KAFKA_HOME/config/server-2.properties
$ kafka-server-start.sh $KAFKA_HOME/config/server-3.properties

查看进程

[hadoop@hadoop001 bin]$ jps
20176 QuorumPeerMain
8432 Kafka
8071 Kafka
7643 Kafka
11101 Jps
[hadoop@hadoop001 bin]$ 

[hadoop@hadoop001 bin]$ jps -m
3232 NameNode
20176 QuorumPeerMain /home/hadoop/app/zookeeper/bin/../conf/zoo.cfg
8432 Kafka ../config/server3.properties
8071 Kafka ../config/server2.properties
7643 Kafka ../config/server1.properties
11199 Jps -m
[hadoop@hadoop001 bin]$ 

创建topic（指定副本数量为3）

[hadoop@hadoop001 bin]$ ./kafka-topics.sh  --creat --zookeeper hadoop001:2181/kafka --replication-factor 3 --partitions 1 --topic leo-offset
Created topic "leo-offset".

查看某个topic的详细信息

[hadoop@hadoop001 bin]$ ./kafka-topics.sh --describe  --zookeeper hadoop001:2181/kafka --topic leo-offset-test
Topic:leo-offset-test   PartitionCount:1        ReplicationFactor:3     Configs:
        Topic: leo-offset-test  Partition: 0    Leader: 1       Replicas: 2,3,1 Isr: 1