Kafka部署

搭建环境

Zookeeper-3.4.6
Scala-2.11
Kafka_2.11-0.10.0.1.
JDK-1.8

下载地址:

Zookeeper:
http://mirror.bit.edu.cn/apache/zookeeper/current/

Scala：
http://www.scala-lang.org/download/2.11.8.html

Kafka:
http://kafka.apache.org/downloads

一.Zookeeper部署

1.下载解压zookeeper-3.4.6.tar.gz
2.修改配置 zoo.cfg

  tickTime=2000
  initLimit=10
  syncLimit=5
  dataDir=/opt/software/zookeeper/data

  clientPort=2181

  server.1=hadoop001:2888:3888
  server.2=hadoop002:2888:3888
  server.3=hadoop003:2888:3888

创建目录并写入myid

zookeeper]#  mkdir data
zookeeper]# touch data/myid
zookeeper]# echo 1 > data/myid

3.hadoop002/003,也修改配置,如下

software]# scp -r  zookeeper 192.168.137.141:/opt/software/
software]# scp -r  zookeeper 192.168.137.142:/opt/software/

zookeeper]# echo 2 > data/myid
zookeeper]# echo 3 > data/myid

###切记不可echo 3 >data/myid,将>前后空格保留,否则无法将 3 写入myid文件

4.启动Zookeeper集群（安装myid中数字的大小开启）

bin]# ./zkServer.sh start
bin]# ./zkServer.sh start
bin]# ./zkServer.sh start

5.查看Zookeeper状态

    bin]# ./zkServer.sh status
    JMX enabled by default
    Using config: /opt/software/zookeeper/bin/../conf/zoo.cfg
    Mode: follower
    bin]#  ./zkServer.sh status
    JMX enabled by default
    Using config: /opt/software/zookeeper/bin/../conf/zoo.cfg
    Mode: leader
    bin]#  ./zkServer.sh status
    JMX enabled by default
    Using config: /opt/software/zookeeper/bin/../conf/zoo.cfg
    Mode: follower

二.Kafka部署

1.解压并配置Scala，设置环境变量

2.下载基于Scala 2.11的kafka版本为0.10.0.1

software]# tar -xzvf kafka_2.11-0.10.0.1.tgz
software]# ln -s kafka_2.11-0.10.0.1 kafka       //软链接

3.创建logs目录和修改server.properties

kafka]# mkdir logs
kafka]# cd config/
config]# vi server.properties
broker.id=1
port=9092
host.name=***.***.***.***
log.dirs=/opt/software/kafka/logs
zookeeper.connect=***.***.***.***:2181,***.***.***.***:2181,***.***.***.***:2181/kafka

4.配置kafka环境变量

5.另外两台机器如上操作

6.启动/停止

kafka]# nohup kafka-server-start.sh config/server.properties &
kafka]# nohup kafka-server-start.sh config/server.properties &
kafka]# nohup kafka-server-start.sh config/server.properties &
###停止
bin/kafka-server-stop.sh

Kafka相关资料

kafka