（8）kafka8

-rw-r–r--. 1 hadoop hadoop 42136632 Oct 18 12:42 kafka_2.11-0.11.0.2.tgz
-rw-r–r--. 1 hadoop hadoop 79630940 Oct 18 12:43 kafka-manager-1.3.3.22.zip

[hadoop@hadoop102 /opt/software]$tar-zxvfkafk{a}_2.11-0.11.0.2.tgz-C/opt/module/[hadoop@hadoop102/opt/module]$mv kafka_2.11-0.11.0.2/ kafka
2.2 Kafka集群部署
1）解压安装包
[atguigu@hadoop102 software]$ tar -zxvf kafka_2.11-0.11.0.2.tgz -C /opt/module/
2）修改解压后的文件名称
[atguigu@hadoop102 module]$ mv kafka_2.11-0.11.0.0/ kafka
3）在/opt/module/kafka目录下创建logs文件夹
[atguigu@hadoop102 kafka]$ mkdir logs
4）修改配置文件
[atguigu@hadoop102 kafka]$ cd config/
[atguigu@hadoop102 config]$ vi server.properties
输入以下内容：
#broker的全局唯一编号，不能重复
broker.id=0
#删除topic功能使能
delete.topic.enable=true
#处理网络请求的线程数量
num.network.threads=3
#用来处理磁盘IO的现成数量
num.io.threads=8
#发送套接字的缓冲区大小
socket.send.buffer.bytes=102400
#接收套接字的缓冲区大小
socket.receive.buffer.bytes=102400
#请求套接字的缓冲区大小
socket.request.max.bytes=104857600
#kafka运行日志存放的路径
log.dirs=/opt/module/kafka/logs
#topic在当前broker上的分区个数
num.partitions=1
#用来恢复和清理data下数据的线程数量
num.recovery.threads.per.data.dir=1
#segment文件保留的最长时间，超时将被删除
log.retention.hours=168
#配置连接Zookeeper集群地址
zookeeper.connect=hadoop102:2181,hadoop103:2181,hadoop104:2181
5）配置环境变量
[atguigu@hadoop102 module]$ sudo vi /etc/profile

#KAFKA_HOME
export KAFKA_HOME=/opt/module/kafka
export PATH=$PATH:$KAFKA_HOME/bin

[atguigu@hadoop102 module]$ source /etc/profile
6）分发安装包
[atguigu@hadoop102 module]$ xsync kafka/
注意：分发之后记得配置其他机器的环境变量
7）分别在hadoop103和hadoop104上修改配置文件/opt/module/kafka/config/server.properties中的broker.id=1、broker.id=2
注：broker.id不得重复
8）启动集群
依次在hadoop102、hadoop103、hadoop104节点上启动kafka
[atguigu@hadoop102 kafka]$ bin/kafka-server-start.sh config/server.properties &
[atguigu@hadoop103 kafka]$ bin/kafka-server-start.sh config/server.properties &
[atguigu@hadoop104 kafka]$ bin/kafka-server-start.sh config/server.properties &
9）关闭集群
[atguigu@hadoop102 kafka]$ bin/kafka-server-stop.sh stop
[atguigu@hadoop103 kafka]$ bin/kafka-server-stop.sh stop
[atguigu@hadoop104 kafka]$ bin/kafka-server-stop.sh stop
集群无法关闭解决
[hadoop@hadoop102 /opt/module/kafka/bin]KaTeX parse error: Expected 'EOF', got '#' at position 27: …erver-stop.sh #̲PIDS=(ps ax | grep -i ‘kafka.Kafka’ | grep java | grep -v grep | awk '{print KaTeX parse error: Expected 'EOF', got '}' at position 2: 1}̲') PIDS=(ps -ef |grep kafka |grep -v grep| awk ‘{print $2}’)

4.5.2Kafka集群启动停止脚本

1）在/home/atguigu/bin目录下创建脚本kf.sh
[atguigu@hadoop102 bin]$ vim kf.sh
在脚本中填写如下内容
#! /bin/bash

case $1 in
“start”){
for i in hadoop102 hadoop103 hadoop104
do
echo " --------启动 $i Kafka-------"
# 用于KafkaManager监控
ssh $i “export JMX_PORT=9988&& /opt/module/kafka/bin/kafka-server-start.sh -daemon /opt/module/kafka/config/server.properties "
done
};;
“stop”){
for i in hadoop102 hadoop103 hadoop104
do
echo " --------停止 $i Kafka-------”
ssh KaTeX parse error: Expected 'EOF', got '}' at position 66: …" done }̲;; esac 说明：启动Ka… chmod 777 kf.sh
3）kf集群启动脚本
[atguigu@hadoop102 module]$ kf.sh start
4）kf集群停止脚本
[atguigu@hadoop102 module]$ kf.sh stop