Linux-Kafka 3.7.0 Kraft+SASL认证模式集群安装与部署超详细

即将雄起的运维玩家

已于 2024-07-05 09:14:08 修改

阅读量4.5k

点赞数 24

文章标签： linux kafka 运维

于 2024-07-04 17:40:51 首次发布

本文链接：https://blog.csdn.net/qq_41118173/article/details/140183152

版权

1.集群规划

一般模式下，元数据在 zookeeper 中，运行时动态选举 controller，由controller 进行 Kafka 集群管理。kraft 模式架构（实验性）下，不再依赖 zookeeper 集群，而是用三台 controller 节点代替 zookeeper，元数据保存在 controller 中，由 controller 直接进行 Kafka 集群管理。

好处有以下几个：

Kafka 不再依赖外部框架，而是能够独立运行
controller 管理集群时，不再需要从 zookeeper 中先读取数据，集群性能上升
由于不依赖 zookeeper，集群扩展时不再受到 zookeeper 读写能力限制
controller 不再动态选举，而是由配置文件规定。可以有针对性的加强controller 节点的配置，而不是像以前一样对随机 controller 节点的高负载束手无策。

kfka1 192.172.21.120)	kfka2 192.172.21.121)	kfka3 192.172.21.122)
kafka	kafka	kafka

2.集群部署

1.下载kafka二进制包

https://kafka.apache.org/downloads

2.解压

tar -zxvf /data/kafka_2.13-3.7.0.tgz

3.修改配置文件(kfka1 192.172.21.120上节点的配置为例)

cd /usr/kafka/kafka_2.13-3.7.0/config/kraft
vi server.properties

注：Kraft模式的配置文件在config目录的kraft子目录下

# Licensed to the Apache Software Foundation (ASF) under one or more
# contributor license agreements. See the NOTICE file distributed with
# this work for additional information regarding copyright ownership.
# The ASF licenses this file to You under the Apache License, Version 2.0
# (the "License"); you may not use this file except in compliance with
# the License. You may obtain a copy of the License at
#
# http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.

#
# This configuration file is intended for use in KRaft mode, where
# Apache ZooKeeper is not present.
#

############################# Server Basics #############################

# The role of this server. Setting this puts us in KRaft mode
#角色
process.roles=broker,controller

# The node id associated with this instance's roles
#id
node.id=1

# The connect string for the controller quorum
controller.quorum.voters=1@192.172.21.120:19093,2@192.172.21.121:19093,3@192.172.21.122:19093

############################# Socket Server Settings #############################

# The address the socket server listens on.
# Combined nodes (i.e. those with `process.roles=broker,controller`) must list the controller listener here at a minimum.
# If the broker listener is not defined, the default listener will use a host name that is equal to the value of java.net.InetAddress.getCanonicalHostName(),
# with PLAINTEXT listener name, and port 19092.
# FORMAT:
# listeners = listener_name://host_name:port
# EXAMPLE:
# li