kafka2.8.2集群离线安装

版本：Centos7、kafka_2.13-2.8.2
节点：
27.196.38.200 kafka1
27.196.38.201 kafka2
27.196.38.202 kafka3

基础配置

在所有节点操作

[root@localhost ~]# systemctl stop firewalld
[root@localhost ~]# systemctl disable firewalld
Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service.
Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.
[root@localhost ~]# setenforce 0
[root@localhost ~]# sed -i "s/SELINUX=enforcing/SELINUX=disabled/g" /etc/selinux/config

安装JDK1.8(略)

安装zookeeper

所有节点执行

tar -zxvf apache-zookeeper-3.8.0-bin.tar.gz
mv apache-zookeeper-3.8.0-bin /usr/local/zookeeper-3.8.0
cd /usr/local/zookeeper-3.8.0/conf
cp zoo_sample.cfg zoo.cfg
mkdir /data/zkdata

vi zoo.cfg
dataDir=/data/zkdata
server.1=27.196.38.200:2888:3888
server.2=27.196.38.201:2888:3888
server.3=27.196.38.202:2888:3888

新建对应不同server的myid，和zookeeper.properties中的值相同:
27.196.38.200:

echo 1 >/data/zkdata/myid

27.196.38.201:

echo 2 >/data/zkdata/myid

27.196.38.202:

echo 3 >/data/zkdata/myid

# 启动zookeeper
/usr/local/zookeeper-3.8.0/bin/zkServer.sh start -daemon > /tmp/zklog.txt 2>&1 &
# 查看zookeeper状态
/usr/local/zookeeper-3.8.0/bin/zkServer.sh status
# 关闭zookeeper服务
/usr/local/zookeeper-3.8.0/bin/zkServer.sh stop

设置服务和开机启动

vi /usr/lib/systemd/system/zookeeper.service
[Unit]
Description=zookeeper-3.8.0
After=syslog.target network.target
[Service]
Restart=alwaysRestartSec=1
Type=forking
TimeoutSec=120
User=root
ExecStart=/usr/local/zookeeper-3.8.0/bin/zkServer.sh start
ExecStop=/usr/local/zookeeper-3.8.0/bin/zkServer.sh stop
ExecReload=/usr/local/zookeeper-3.8.0/bin/zkServer.sh restart
Restart=on-failure
[Install]
WantedBy=multi-user.target

systemctl daemon-reload
systemctl enable zookeeper.service

# 启动
systemctl start zookeeper
# 查看状态
systemctl status zookeeper
# 停止
systemctl stop zookeeper

安装kafka

安装

tar -zxvf kafka_2.13-3.3.1.tar
mv kafka_2.13-3.3.1 /usr/local/kafka
mkdir -p /data/kafka

修改config/server.properties配置文件

vi /usr/local/kafka/config/server.properties

27.196.38.200配置：

broker.id=0
listeners=PLAINTEXT://27.196.38.200:9092
delete.topic.enable=true
log.dirs=/data/kafka
zookeeper.connect=27.196.38.200:2181,27.196.38.201:2181,27.196.38.202:2181

27.196.38.201配置：

broker.id=1
listeners=PLAINTEXT://27.196.38.201:9092
delete.topic.enable=true
log.dirs=/data/kafka
zookeeper.connect=27.196.38.200:2181,27.196.38.201:2181,27.196.38.202:2181

27.196.38.202配置：

broker.id=2
listeners=PLAINTEXT://27.196.38.202:9092
delete.topic.enable=true
log.dirs=/data/kafka
zookeeper.connect=27.196.38.200:2181,27.196.38.201:2181,27.196.38.202:2181

启动kafka：

/usr/local/kafka/bin/kafka-server-start.sh -daemon /usr/local/kafka/config/server.properties > /tmp/kafkalog.txt 2>&1 &

查看启动日志

tail -f /tmp/kafkalog.txt

kafka停止命令：

/usr/local/kafka/bin/kafka-server-stop.sh 

创建systemd服务

vi /usr/lib/systemd/system/kafka.service 
[Unit]
Description=kafka.service
After=network.target zookeeper.service
[Service]
Type=simple
Environment=PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/root/bin:/usr/java/jdk1.8.0_211-amd64/bin:/usr/java/jdk1.8.0_211-amd64/jre/bin
ExecStart=/usr/local/kafka/bin/kafka-server-start.sh /usr/local/kafka/config/server.properties
ExecStop=/usr/local/kafka/bin/kafka-server-stop.sh
Restart=on-failure
[Install]
WantedBy=multi-user.target

重新加载并启动

systemctl daemon-reload
systemctl enable kafka.service
systemctl start kafka.service
systemctl status kafka.service -l

创建Topic：

/usr/local/kafka/bin/kafka-topics.sh --create --bootstrap-server 27.196.38.200:9092 --replication-factor 3 --partitions 3 --topic cicdlog

查看topic list：

/usr/local/kafka/bin/kafka-topics.sh --list --bootstrap-server 27.196.38.200:9092

控制台生产消息：

/usr/local/kafka/bin/kafka-console-producer.sh --broker-list 27.196.38.200:9092 --topic cicdlog

控制台消费消息：

/usr/local/kafka/bin/kafka-topics.sh --create --bootstrap-server 27.196.38.200:9092 --replication-factor 1 --partitions 3 --topic test

kafka manager安装

安装节点：27.196.38.220

1、安装JDK11，cmak3.0.0.6依赖JDK11。

tar -zxvf jdk-11.0.16.1_linux-x64_bin.tar.gz
mv jdk-11.0.16.1 /usr/local/

不配置环境环境，以免影响其它应用。
2、安装cmak
解压cmak-3.0.0.6.zip，拷贝到/usr/local/目录下
修改配置文件

vi /usr/local/cmak-3.0.0.6/conf/application.conf
application.home=/usr/local/cmak-3.0.0.6
cmak.zkhosts="27.196.38.200:2181,27.196.38.201:2181,27.196.38.202:2181"

后台启动

chmod -R 777 /usr/local/cmak-3.0.0.6
nohup /usr/local/cmak-3.0.0.6/bin/cmak -java-home /usr/local/jdk-11.0.16.1 > /tmp/cmaklog.txt 2>&1 &

重启需要删掉 /data/service/cmak-3.0.0.6/RUNNING_PID 文件
访问地址：http://27.196.38.220:9000