kafka集群安装部署

一、安装zookeeper集群

安装部署kafka集群之前，需要安装部署zookeeper集群，可参看我的另一篇博客：https://blog.csdn.net/weixin_42273746/article/details/84563374

kafka也自带zookeeper，不另外安装zookeeper集群，可以使用kafka自带的zookeeper，下文有介绍！

二、安装kafka集群

1.下载

可以去官网下载安装包：http://kafka.apache.org/downloads

2.解压

[root@master ~]# tar -zxvf kafka_2.11-1.1.0.tgz -C /home

3.修改server.properties配置文件

[root@master ~]# vim /home/kafka_2.11-1.1.0/config/server.properties

修改如下：

broker.id=0
host.name=master
advertised.listeners=PLAINTEXT://master:9092
log.dirs=/home/kafka_2.11-1.1.0/logs
zookeeper.connect=master:2181,slave1:2181,slave2:2181

每个节点的broker.id必须不同，我设置的为：master为0；slave1位1，slave2位2。

4.三个节点都需进行相应的配置和修改。

5.kafka常用命令

查看我的另一篇博客：https://blog.csdn.net/weixin_42273746/article/details/84570547

---

若不安装zookeeper集群，可用kafka自带的zookeeper

1.创建目录

在/home/kafka_2.11-1.1.0目录下创建一个zookeeper文件夹，然后在zookeeper文件夹下创建data和log两个文件夹

2.修改配置文件

[root@master ~]# vim /home/kafka_2.11-1.1.0/config/zookeeper.properties

修改为如下内容：

# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements.  See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License.  You may obtain a copy of the License at
# 
#    http://www.apache.org/licenses/LICENSE-2.0
# 
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
# the directory where the snapshot is stored.
dataDir=/home/kafka_2.11-1.1.0/zookeeper/data
dataLogDir=/home/kafka_2.11-1.1.0/zookeeper/log
# the port at which the clients will connect
clientPort=2181
# disable the per-ip limit on the number of connections since this is a non-production config
maxClientCnxns=100
tickTime=2000
initLimit=10
syncLimit=5

server.0=master:2888:3888
server.1=slave1:2888:3888
server.2=slave2:2888:3888

3.创建并编辑myid文件

[root@master ~]# vim /home/kafka_2.11-1.1.0/zookeeper/data/myid

添加如下内容：

0

每个节点的myid都必须不一样。

4.每个节点都需要进行修改和配置

5.启动zookeeper

在kafka的目录下：

[root@master kafaka_2.11-1.1.0]# ./bin/zookeeper-server-start.sh config/zookeeper.properties &