先对Kafka调优
关闭Kafka集群
/home/kafka.sh stop
启停脚本 https://blog.csdn.net/weixin_44371237/article/details/127102413
kafka默认1G内存,Kafka-Eagle不够用,修改kafka-server-start.sh
vim bin/kafka-server-start.sh
修改如下参数值:
if [ "x$KAFKA_HEAP_OPTS" = "x" ]; then
export KAFKA_HEAP_OPTS="-Xmx1G -Xms1G"
fi
为:
if [ "x$KAFKA_HEAP_OPTS" = "x" ]; then
export KAFKA_HEAP_OPTS="-server -Xms2G -Xmx2G -XX:PermSize=128m -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -XX:ParallelGCThreads=8 -XX:ConcGCThreads=5 -XX:InitiatingHeapOccupancyPercent=70"
export JMX_PORT="9999"
#export KAFKA_HEAP_OPTS="-Xmx1G -Xms1G"
fi
分发到其他服务器
/home/xsync /opt/module/kafka/bin/kafka-server-start.sh
分发脚本 https://blog.csdn.net/weixin_44371237/article/details/127102600
启动Kafka集群
/home/kafka.sh start
安装kafka-eagle
官网下载 https://www.kafka-eagle.org/
上传kafka-eagle-bin-2.0.8.tar.gz到服务器
先解压到本目录查看有什么内容
tar -zxf kafka-eagle-bin-2.0.8.tar.gz
再解压efak-web-2.0.8-bin.tar.gz到安装目录
tar -zxf efak-web-2.0.8-bin.tar.gz -C /opt/module
进入/opt/module,修改名称
mv efak-web-2.0.8 efak
修改系统配置
vim /opt/module/efak/conf/system-config.properties
注释
#efak.zk.cluster.alias=cluster1,cluster2
#cluster1.zk.list=tdn1:2181,tdn2:2181,tdn3:2181
#cluster2.zk.list=xdn10:2181,xdn11:2181,xdn12:2181
添加
efak.zk.cluster.alias=cluster1
cluster1.zk.list=hadoop100:2181,hadoop101:2181,hadoop102:2181/kafka
注释
#cluster2.efak.offset.storage=zk
注释
#efak.driver=org.sqlite.JDBC
#efak.url=jdbc:sqlite:/hadoop/kafka-eagle/db/ke.db
#efak.username=root
#efak.password=www.kafka-eagle.org
添加
efak.driver=com.mysql.cj.jdbc.Driver
efak.url=jdbc:mysql://hadoop100:3306/ke?useUnicode=true&characterEncoding=UTF-8&zeroDateTimeBehavior=convertToNull
efak.username=root
efak.password=Cxb#1987
配置环境变量
vim /etc/profile.d/my_env.sh
# kafkaEFAK
export KE_HOME=/opt/module/efak
export PATH=$PATH:$KE_HOME/bin
source /etc/profile
启动
bin/ke.sh start
按提示登录
* Welcome, Now you can visit 'http://192.168.56.100:8048'
* Account:admin ,Password:123456