Apache-Kafka集群安装

kafka安装记录
1、准备环境
使用VMware安装3台Linux服务器

mini版本
hostname 依次为 node01、node02、node03
关闭防火墙
每台机器都操作一遍
service iptables stop && chkconfig iptables off
配置网卡
配置/etc/hosts文件

192.168.140.128 node01 zk01 kafka01
192.168.140.129 node02 zk02 kafka02
192.168.140.130 node03 zk03 kafka03
scp /etc/hosts node02:/etc/
scp /etc/hosts node03:/etc/

配置Yum

yum install -y lrzsz
yum install -y wget
mv /etc/yum.repos.d/CentOS-Base.repo /etc/yum.repos.d/CentOS-Base.repo.backup
cd /etc/yum.repos.d
wget http://mirrors.163.com/.help/CentOS6-Base-163.repo
yum clean all && yum makecache

配置免密登录

yum -y install openssh-clients
ssh-keygen 四个回车
ssh-copy-id node1
ssh-copy-id node2
ssh-copy-id node3

2、安装JDK&ZK&KAFKA
安装JDK

mkdir -p /export/servers
mkdir -p /export/software
mkdir -p /export/logs
mkdir -p /export/data
mkdir -p /export/data/zk
mkdir -p /export/data/kafka
mkdir -p /export/logs/zk
-
cd /export/software/
rz 选择 jdk-8u141-linux-x64.tar.gz
tar -zxvf jdk-8u141-linux-x64.tar.gz -C ../servers/
cd ../servers/
mv jdk1.8.0_141 jdk
source /etc/profile
-
export JAVA_HOME=/export/servers/jdk
export JRE_HOME=${JAVA_HOME}/jre
export CLASSPATH=.:${JAVA_HOME}/lib:${JRE_HOME}/lib
export PATH=${JAVA_HOME}/bin:$PATH
-
scp -r jdk node02:/export/servers/
scp -r jdk node03:/export/servers/
scp /etc/profile node02:/etc/
scp /etc/profile node03:/etc/
-
source /etc/profile on node01、node02、node03
java -version on node01 on node01、node02、node03

安装zookeeper

cd /export/software/
wget    http://219.238.7.73/files/703900000A354B91/apache.fayea.com/zookeeper/zookeeper-3.4.9/zookeeper-3.4.9.tar.gz
tar -zxvf zookeeper-3.4.9.tar.gz -C ../servers/
cd ../servers/
mv zookeeper-3.4.9/ zk
-
scp -r zk node02:/export/servers/
scp -r zk node03:/export/servers/
-
touch /export/data/zk/myid on node01、node02、node03
echo 1 > /export/data/zk/myid on node01
echo 2 > /export/data/zk/myid on node02
echo 3 > /export/data/zk/myid on node03
-
cd /export/servers/zk/conf/
touch zoo.cfg 
vi zoo.cfg
-
tickTime=2000
initLimit=10
syncLimit=5
dataDir=/export/data/zk
dataLogDir=/export/logs/zk
clientPort=2181
server.1=node01:2887:3887
server.2=node02:2887:3887
server.3=node03:2887:3887
-
scp zoo.cfg node02:$PWD
scp zoo.cfg node03:$PWD
-
-
zkServer.sh start on node01、node02、node03
zkServer.sh status on node01、node02、node03

安装Kafka

cd /export/software/
wget http://219.238.7.67/files/518200000AE89181/mirror.bit.edu.cn/apache/kafka/1.0.0/kafka_2.11-1.0.0.tgz
tar -zxvf kafka_2.11-1.0.0.tgz -C ../servers/
cd ../servers/
mv kafka_2.11-1.0.0 kafka
-
scp -r kafka node02:/export/servers/
scp -r kafka node03:/export/servers/
-
cd /export/servers/kafka/config/
-
修改配置文件
cd /export/servers/kafka/
nohup bin/kafka-server-start.sh config/server.properties & on node01、node02、node03

在window使用工具查看kafka集群

首先使用hosts.exe工具配置本地host

192.168.140.128 node01 zk01 kafka01
192.168.140.129 node02 zk02 kafka02
192.168.140.130 node03 zk03 kafka03
zookeeper_java_client.zip

解压、链接即可。