kafka分布式集群安装

kafka分布式集群安装

一、zookeeper

1.下载并解压缩

http://zookeeper.apache.org/官网下载并解压
tar -zxf zookeeper-3.4.8.tar.gz -C /usr/local/

2.配置环境变量

zookeeper environment

export ZOOKEEPER_HOME=/usr/local/zookeeper-3.4.8
export PATH=$ZOOKEEPE{R}_{H}OME/bin:$PATH

3.对zookeeper进行配置

添加配置文件conf/zoo.cfg
dataDir=/usr/local/zookeeper-3.4.8/data
server.0=spark1:2888:3888
server.1=spark2:2888:3888
server.2=spark3:2888:3888
分别在spark1、spark2和spark3中
cd /usr/local/zookeeper-3.4.8 ;mkdir data;echo “0”>myid
cd /usr/local/zookeeper-3.4.8 ;mkdir data;echo “1”>>myid
cd /usr/local/zookeeper-3.4.8 ;mkdir data;echo “2”>myid
其中上边的数字0，1，2与conf/zoo.cfg中的server.*一致

4.启动

zkServer.sh start
zkServer.sh status
zkCli.sh

二、kafka

1.下载解压缩

http://kafka.apache.org/downloads

2.环境变量

kafa environment

export KAFKA_HOME=/usr/local/kafka_2.11-1.0.0
export PATH=$ZOOKEEPER_HOME/bin:$PATH

3.config/server.properties

broker.id=0
zookeeper.connect=192.168.6.137:2181, 192.168.6.138:2181, 192.168.6.139:2181

4.启动kafka集群即可（三台机器）

$KAFKA_HOME/bin/kafka-server-start.sh -daemon $KAFKA_HOME/config/server.properties