Kafka监控eagle

最新推荐文章于 2024-07-30 11:28:03 发布

daydayup9527

最新推荐文章于 2024-07-30 11:28:03 发布

阅读量240

点赞数

分类专栏： kafka 文章标签：学习 kubernetes 运维 linux

本文链接：https://blog.csdn.net/weixin_60092693/article/details/124365252

版权

kafka 专栏收录该内容

4 篇文章 0 订阅

订阅专栏

该博客介绍了如何在节点上部署和启动Kafka服务，包括设置JVM参数，并通过`jps`验证进程状态。接着展示了查看Kafka启动日志的方法。然后，博主安装并配置了Kafka监控工具Kafka-Eagle，修改了配置文件以监控指定的Kafka集群，并设置了邮件报警。最后，启动Kafka-Eagle服务并给出了访问URL，同时展示了MySQL数据库中与Kafka-Eagle相关的表格。

摘要由CSDN通过智能技术生成

Kafka 监控

[root@node6 ~]# /opt/zookeeper/bin/zk
[root@node6 ~]# /opt/kafka/bin/kafka-server-start.sh -daemon /opt/kafka/config/server.properties
[root@node6 ~]# jps
1571 Kafka				//节点上都需要启动，都要有

[root@node6 ~]# /opt/kafka/bin/kafka-server-stop.sh   //停掉kafka
[root@node6 ~]# cat /opt/kafka/logs/kafkaServer.out   //启动的日志

[root@node6 eagle]# cat -n /opt/kafka/bin/kafka-server-start.sh
###########kafka 启动命令添加33  34行注释 35行#######################
    32  if [ "x$KAFKA_HEAP_OPTS" = "x" ]; then
    33    export KAFKA_HEAP_OPTS="-server -Xms1G -Xmx1G -XX:PermSize=128m -XX:+UseG1GC -XX:MaxGCPauseMillis=200 -XX:ParallelGCThreads=8 -XX:ConcGCThreads=5 -XX:InitiatingHeapOccupancyPercent=70"
    34   export JMX_PORT="9999"
    35   #export KAFKA_HEAP_OPTS="-Xmx1G -Xms1G"
    36  fi
 [root@node6 ~]# /opt/kafka/bin/kafka-server-start.sh -daemon /opt/kafka/config/server.properties

[root@node6 ~]# tar xf kafka-eagle-web-1.3.7-bin.tar.gz
[root@node6 ~]# mv kafka-eagle-web-1.3.7 /opt/eagle
[root@node6 ~]# cd /opt/eagle/
[root@node6 ~]# ls 
bin  conf  db  font  kms  logs
[root@node6 ~]# chmod +x bin/ke.sh

修改eagle配置文件

[root@node6 eagle]# vim conf/system-config.properties
2:# multi zookeeper&kafka cluster list		#可以监控多个kafka集群
4:kafka.eagle.zk.cluster.alias=cluster1    #定义监控的集群名，多个用,隔开
5:cluster1.zk.list=node4:2181,node5:2181,node6:2181  #集群信息
6:#cluster2.zk.list=xdn10:2181,xdn11:2181,xdn12:2181
9:# zk client thread limit
11:kafka.zk.limit.size=25
14:# kafka eagle webui port
16:kafka.eagle.webui.port=8048		#web端口
19:# kafka offset storage
21:cluster1.kafka.eagle.offset.storage=kafka  #存储在kafka 旧版本在zookeeper里
22:#cluster2.kafka.eagle.offset.storage=zk
25:# enable kafka metrics
27:kafka.eagle.metrics.charts=true    #图表打开
28:kafka.eagle.sql.fix.error=false
31:# kafka sql topic records max
33:kafka.eagle.sql.topic.records.max=5000
36:# alarm email configure
38:kafka.eagle.mail.enable=false
39:kafka.eagle.mail.sa=alert_sa@163.com   #以下暂时默认
40:kafka.eagle.mail.username=alert_sa@163.com
41:kafka.eagle.mail.password=mqslimczkdqabbbh
42:kafka.eagle.mail.server.host=smtp.163.com
43:kafka.eagle.mail.server.port=25
...
78:# kafka jdbc driver address
80:kafka.eagle.driver=com.mysql.jdbc.Driver
#######数据地址192.168.1.17:3306用户eagel 密码，
81:kafka.eagle.url=jdbc:mysql://192.168.1.17:3306/ke?useUnicode=true&ch   
82:aracterEncoding=UTF-8&zeroDateTimeBehavior=convertToNull
83:kafka.eagle.username=eagle
84:kafka.eagle.password=123456

MariaDB [(none)]> grant all on *.* to eagle@"%" identified by "123456";

[root@node6 eagle]# bin/ke.sh start
Welcome to     kafka eagle
Version 1.3.7
*******************************************************************
* Kafka Eagle Service has started success.
* Welcome, Now you can visit 'http://192.168.1.16:8048/ke'
* Account:admin ,Password:123456

浏览器访问 http://192.168.1.16:8048/ke

[root@node7 ~]# mysql -uroot -p123456
MariaDB [eagle]> use ke;
MariaDB [ke]> show tables;
+------------------+
| Tables_in_ke     |
+------------------+
| ke_alarm         |
| ke_clusters      |
| ke_consumer      |
| ke_metrics       |
| ke_p_role        |
| ke_resources     |
| ke_role_resource |
| ke_sql_history   |
| ke_topic_rank    |
| ke_trend         |
| ke_user_role     |
| ke_users         |
+------------------+

在这里插入图片描述