大数据之 Kafka 监控完整使用 (第五章)

最新推荐文章于 2023-09-19 12:00:00 发布

小坏讲微服务

最新推荐文章于 2023-09-19 12:00:00 发布

阅读量1.3k

点赞数 1

分类专栏：小坏将大数据 (kafka) 第五阶段文章标签： kafka big data java

本文链接：https://blog.csdn.net/qq_42082701/article/details/121209004

版权

小坏将大数据 (kafka) 第五阶段专栏收录该内容

7 篇文章 1 订阅

订阅专栏

大数据之 Kafka 监控完整使用

一、Kafka Eagle

一、Kafka Eagle

1.修改kafka 启动命令

修改 kafka-server-start.sh 命令中

if [ "x$KAFKA_HEAP_OPTS" = "x" ]; then
 export KAFKA_HEAP_OPTS="-Xmx1G -Xms1G"
fi

为

if [ "x$KAFKA_HEAP_OPTS" = "x" ]; then
 export KAFKA_HEAP_OPTS="-server -Xms2G -Xmx2G -XX:PermSize=128m 
-XX:+UseG1GC -XX:MaxGCPauseMillis=200 -XX:ParallelGCThreads=8 -
XX:ConcGCThreads=5 -XX:InitiatingHeapOccupancyPercent=70"
 export JMX_PORT="9999"
 #export KAFKA_HEAP_OPTS="-Xmx1G -Xms1G"
fi

注意：修改之后在启动 Kafka 之前要分发之其他节点

2.上传压缩包

上传压缩包kafka-eagle-bin-1.3.7.tar.gz 到集群/opt/software 目录

3.解压到本地

[atguigu@hadoop102 software]$ tar -zxvf kafka-eagle-bin-1.3.7.tar.gz

4.进入刚才解压的目录

[atguigu@hadoop102 kafka-eagle-bin-1.3.7]$ ll
总用量 82932
-rw-rw-r–. 1 atguigu atguigu 84920710 8 月 13 23:00 kafka-eagleweb-1.3.7-bin.tar.gz

5.将 kafka-eagle-web-1.3.7-bin.tar.gz 解压至/opt/module

[atguigu@hadoop102 kafka-eagle-bin-1.3.7]$ tar -zxvf kafka-eagleweb-1.3.7-bin.tar.gz -C /opt/module/

6.修改名称

[atguigu@hadoop102 module]$ mv kafka-eagle-web-1.3.7/ kafka-web

7.给启动文件执行权限

[atguigu@hadoop102 eagle]$ cd bin/
[atguigu@hadoop102 bin]$ ll
总用量 12
-rw-r–r--. 1 atguigu atguigu 1848 8 月 22 2017 ke.bat
-rw-r–r--. 1 atguigu atguigu 7190 7 月 30 20:12 ke.sh
[atguigu@hadoop102 bin]$ chmod 777 ke.sh

8.修改配置文件

######################################
# multi zookeeper&kafka cluster list
######################################
kafka.eagle.zk.cluster.alias=cluster1
cluster1.zk.list=hadoop102:2181,hadoop103:2181,hadoop104:2181
######################################
# kafka offset storage
######################################
cluster1.kafka.eagle.offset.storage=kafka
######################################
# enable kafka metrics
######################################
kafka.eagle.metrics.charts=true
kafka.eagle.sql.fix.error=false
######################################
# kafka jdbc driver address
######################################
kafka.eagle.driver=com.mysql.jdbc.Driver
kafka.eagle.url=jdbc:mysql://hadoop102:3306/ke?useUnicode=true&ch
aracterEncoding=UTF-8&zeroDateTimeBehavior=convertToNull
kafka.eagle.username=root
kafka.eagle.password=000000

9.添加环境变量

export KE_HOME=/opt/module/kafka-web
export PATH= $P A T H :$ KE_HOME/bin

注意：source /etc/profile

10.启动

[atguigu@hadoop102 kafka-web]$ bin/ke.sh start

[2021-11-11 11:37:01] INFO: Starting  kafka eagle environment check ...
  已创建: META-INF/
  已解压: META-INF/MANIFEST.MF
  已创建: WEB-INF/
  已创建: WEB-INF/classes/
  已创建: WEB-INF/classes/org/
  已创建: WEB-INF/classes/org/smartloli/
  已创建: WEB-INF/classes/org/smartloli/kafka/
  已创建: WEB-INF/classes/org/smartloli/kafka/eagle/
  已创建: WEB-INF/classes/org/smartloli/kafka/eagle/web/
  已创建: WEB-INF/classes/org/smartloli/kafka/eagle/web/dao/
  已创建: WEB-INF/classes/org/smartloli/kafka/eagle/web/controller/
  已创建: WEB-INF/classes/org/smartloli/kafka/eagle/web/sso/
  已创建: WEB-INF/classes/org/smartloli/kafka/eagle/web/sso/filter/
  已创建: WEB-INF/classes/org/smartloli/kafka/eagle/web/sso/pojo/
  已创建: WEB-INF/classes/org/smartloli/kafka/eagle/web/quartz/
  已创建: WEB-INF/classes/org/smartloli/kafka/eagle/web/service/
  已创建: WEB-INF/classes/org/smartloli/kafka/eagle/web/service/impl/
  已创建: WEB-INF/classes/org/smartloli/kafka/eagle/web/pojo/
  已创建: WEB-INF/lib/
  已创建: WEB-INF/views/
  已创建: WEB-INF/views/metrics/
  已创建: WEB-INF/views/cluster/
  已创建: WEB-INF/views/topic/
  已创建: WEB-INF/views/system/
  已创建: WEB-INF/views/consumers/
  已创建: WEB-INF/views/public/
  已创建: WEB-INF/views/alarm/
  已创建: WEB-INF/views/account/
  已创建: WEB-INF/views/main/
  已创建: WEB-INF/views/error/
  已创建: media/
  已创建: media/css/
  已创建: media/css/plugins/
  已创建: media/css/plugins/datatimepicker/
  已创建: media/css/plugins/select2/
  已创建: media/css/img/
  已创建: media/css/public/
  已创建: media/css/public/images/
  已创建: media/css/public/account/
  已创建: media/css/fonts/
  已创建: media/js/
  已创建: media/js/plugins/
  已创建: media/js/plugins/datatimepicker/
  已创建: media/js/plugins/codemirror/
  已创建: media/js/plugins/echart/
  已创建: media/js/plugins/datatables/
  已创建: media/js/plugins/terminal/
  已创建: media/js/plugins/d3/
  已创建: media/js/plugins/tokenfield/
  已创建: media/js/plugins/select2/
  已创建: media/js/plugins/magicsuggest/
  已创建: media/js/public/
  已创建: media/js/main/
  已创建: media/js/main/metrics/
  已创建: media/js/main/cluster/
  已创建: media/js/main/topic/
  已创建: media/js/main/system/
  已创建: media/js/main/alarm/
  已创建: media/js/main/consumer/
  已创建: media/js/main/account/
  已创建: media/js/main/error/
  已创建: media/img/
  已解压: WEB-INF/classes/spring-shiro.xml
  已解压: WEB-INF/classes/metrics-quartz.xml
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/dao/RoleDao.xml
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/dao/MBeanDao.xml
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/dao/UserDao.class
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/dao/MBeanDao.class
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/dao/UserDao.xml
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/dao/AlertDao.class
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/dao/ResourcesDao.xml
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/dao/RoleDao.class
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/dao/TopicDao.class
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/dao/TopicDao.xml
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/dao/AlertDao.xml
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/dao/ResourcesDao.class
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/controller/ResourcesController.class
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/controller/DashboardController.class
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/controller/StartupListener$RunTask.class
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/controller/ConsumersController.class
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/controller/TopicController.class
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/controller/BaseController.class
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/controller/AccountController.class
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/controller/ErrorPageController.class
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/controller/RoleController.class
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/controller/ClusterController.class
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/controller/StartupListener$ContextSchema.class
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/controller/OffsetController.class
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/controller/MetricsController.class
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/controller/AlarmController.class
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/controller/StartupListener.class
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/sso/filter/SSOFilter.class
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/sso/filter/SSORealm.class
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/sso/pojo/SSOAuthenticationToken.class
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/sso/pojo/Resources.class
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/quartz/AlertQuartz$Consumer.class
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/quartz/MetricsQuartz.class
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/quartz/MBeanQuartz.class
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/quartz/AlertQuartz.class
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/quartz/TopicRankQuartz.class
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/quartz/AlertQuartz$Cluster.class
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/service/impl/MetricsServiceImpl.class
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/service/impl/DashboardServiceImpl.class
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/service/impl/RoleServiceImpl.class
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/service/impl/ConsumerServiceImpl.class
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/service/impl/ClusterServiceImpl.class
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/service/impl/ResourceServiceImpl.class
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/service/impl/AlertServiceImpl.class
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/service/impl/OffsetServiceImpl.class
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/service/impl/AccountServiceImpl.class
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/service/impl/TopicServiceImpl.class
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/service/DashboardService.class
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/service/RoleService.class
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/service/AlertService.class
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/service/AccountService.class
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/service/OffsetService.class
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/service/ClusterService.class
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/service/ConsumerService.class
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/service/MetricsService.class
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/service/TopicService.class
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/service/ResourceService.class
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/pojo/Role.class
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/pojo/RoleResource.class
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/pojo/UserRole.class
  已解压: WEB-INF/classes/org/smartloli/kafka/eagle/web/pojo/Signiner.class
  已解压: WEB-INF/classes/shiro-ehcache.xml
  已解压: WEB-INF/classes/alert-quartz.xml
  已解压: WEB-INF/classes/spring-mvc.xml
  已解压: WEB-INF/classes/spring-mybatis.xml
  已解压: WEB-INF/classes/topic-quartz.xml
  已解压: WEB-INF/classes/mbean-quartz.xml
  已解压: WEB-INF/lib/slf4j-api-1.6.6.jar
  已解压: WEB-INF/lib/protobuf-java-3.0.0-beta-1.jar
  已解压: WEB-INF/lib/mail-1.4.7.jar
  已解压: WEB-INF/lib/guava-19.0.jar
  已解压: WEB-INF/lib/spring-context-4.1.6.RELEASE.jar
  已解压: WEB-INF/lib/junit-3.8.1.jar
  已解压: WEB-INF/lib/jopt-simple-5.0.4.jar
  已解压: WEB-INF/lib/jsr305-1.3.9.jar
  已解压: WEB-INF/lib/activation-1.1.jar
  已解压: WEB-INF/lib/aopalliance-1.0.jar
  已解压: WEB-INF/lib/spring-web-4.1.6.RELEASE.jar
  已解压: WEB-INF/lib/h2-1.4.195.jar
  已解压: WEB-INF/lib/mybatis-3.2.6.jar
  已解压: WEB-INF/lib/httpcore-4.4.10.jar
  已解压: WEB-INF/lib/jstl-1.2.jar
  已解压: WEB-INF/lib/commons-httpclient-3.0.jar
  已解压: WEB-INF/lib/slf4j-log4j12-1.7.5.jar
  已解压: WEB-INF/lib/jline-0.9.94.jar
  已解压: WEB-INF/lib/commons-logging-1.1.2.jar
  已解压: WEB-INF/lib/kafka-eagle-api-1.3.7.jar
  已解压: WEB-INF/lib/spring-expression-4.1.6.RELEASE.jar
  已解压: WEB-INF/lib/lucene-core-5.5.2.jar
  已解压: WEB-INF/lib/scala-library-2.11.12.jar
  已解压: WEB-INF/lib/commons-pool-1.3.jar
  已解压: WEB-INF/lib/spring-webmvc-4.1.6.RELEASE.jar
  已解压: WEB-INF/lib/zookeeper-3.4.8.jar
  已解压: WEB-INF/lib/lucene-sandbox-5.5.2.jar
  已解压: WEB-INF/lib/httpclient-4.5.6.jar
  已解压: WEB-INF/lib/shiro-core-1.3.2.jar
  已解压: WEB-INF/lib/log4j-1.2.17.jar
  已解压: WEB-INF/lib/spring-aop-4.1.6.RELEASE.jar
  已解压: WEB-INF/lib/shiro-web-1.3.2.jar
  已解压: WEB-INF/lib/quartz-2.2.1.jar
  已解压: WEB-INF/lib/kafka_2.11-2.0.0.jar
  已解压: WEB-INF/lib/servlet-api-2.5.jar
  已解压: WEB-INF/lib/spring-oxm-4.1.6.RELEASE.jar
  已解压: WEB-INF/lib/jackson-annotations-2.9.0.jar
  已解压: WEB-INF/lib/fastjson-1.2.7.jar
  已解压: WEB-INF/lib/xml-apis-1.0.b2.jar
  已解压: WEB-INF/lib/pentaho-aggdesigner-algorithm-5.1.5-jhyde.jar
  已解压: WEB-INF/lib/spring-test-4.1.6.RELEASE.jar
  已解压: WEB-INF/lib/annotations-13.0.jar
  已解压: WEB-INF/lib/commons-dbcp-1.2.2.jar
  已解压: WEB-INF/lib/shiro-spring-1.3.2.jar
  已解压: WEB-INF/lib/cache-api-1.0.0.jar
  已解压: WEB-INF/lib/metrics-core-2.2.0.jar
  已解压: WEB-INF/lib/commons-io-2.4.jar
  已解压: WEB-INF/lib/kafka-eagle-common-1.3.7.jar
  已解压: WEB-INF/lib/janino-2.7.6.jar
  已解压: WEB-INF/lib/spring-core-4.1.6.RELEASE.jar
  已解压: WEB-INF/lib/calcite-core-1.6.0.jar
  已解压: WEB-INF/lib/druid-1.0.31.jar
  已解压: WEB-INF/lib/snappy-java-1.1.7.1.jar
  已解压: WEB-INF/lib/aspectjweaver-1.8.10.jar
  已解压: WEB-INF/lib/netty-3.7.0.Final.jar
  已解压: WEB-INF/lib/calcite-avatica-1.6.0.jar
  已解压: WEB-INF/lib/scala-logging_2.11-3.9.0.jar
  已解压: WEB-INF/lib/kafka-eagle-core-1.3.7.jar
  已解压: WEB-INF/lib/spring-beans-4.1.6.RELEASE.jar
  已解压: WEB-INF/lib/commons-codec-1.2.jar
  已解压: WEB-INF/lib/dom4j-1.6.1.jar
  已解压: WEB-INF/lib/aspectjrt-1.8.10.jar
  已解压: WEB-INF/lib/commons-beanutils-1.8.3.jar
  已解压: WEB-INF/lib/lucene-queryparser-5.5.2.jar
  已解压: WEB-INF/lib/c3p0-0.9.1.1.jar
  已解压: WEB-INF/lib/spring-tx-4.1.6.RELEASE.jar
  已解压: WEB-INF/lib/ehcache-core-2.5.3.jar
  已解压: WEB-INF/lib/calcite-linq4j-1.6.0.jar
  已解压: WEB-INF/lib/kafka-clients-2.0.0.jar
  已解压: WEB-INF/lib/lucene-queries-5.5.2.jar
  已解压: WEB-INF/lib/gson-2.2.4.jar
  已解压: WEB-INF/lib/commons-lang-2.6.jar
  已解压: WEB-INF/lib/spring-jdbc-4.1.6.RELEASE.jar
  已解压: WEB-INF/lib/sqlite-jdbc-3.21.0.1.jar
  已解压: WEB-INF/lib/spring-context-support-4.1.6.RELEASE.jar
  已解压: WEB-INF/lib/mysql-connector-java-5.1.30.jar
  已解压: WEB-INF/lib/eigenbase-properties-1.1.5.jar
  已解压: WEB-INF/lib/lz4-java-1.4.1.jar
  已解压: WEB-INF/lib/ignite-spring-2.3.0.jar
  已解压: WEB-INF/lib/mybatis-spring-1.2.2.jar
  已解压: WEB-INF/lib/ignite-core-2.3.0.jar
  已解压: WEB-INF/lib/zkclient-0.9.jar
  已解压: WEB-INF/lib/lucene-analyzers-common-5.5.2.jar
  已解压: WEB-INF/lib/scala-reflect-2.11.12.jar
  已解压: WEB-INF/lib/jackson-core-2.9.6.jar
  已解压: WEB-INF/lib/ignite-shmem-1.0.0.jar
  已解压: WEB-INF/lib/jackson-databind-2.9.6.jar
  已解压: WEB-INF/lib/commons-compiler-2.7.6.jar
  已解压: WEB-INF/lib/ignite-indexing-2.3.0.jar
  已解压: WEB-INF/lib/shiro-ehcache-1.3.2.jar
  已解压: WEB-INF/lib/commons-lang3-3.5.jar
  已解压: WEB-INF/lib/kafka-eagle-plugin-1.3.7.jar
  已解压: WEB-INF/web.xml
  已解压: WEB-INF/views/metrics/kafka.jsp
  已解压: WEB-INF/views/metrics/brokers.jsp
  已解压: WEB-INF/views/metrics/zk.jsp
  已解压: WEB-INF/views/cluster/zkcli.jsp
  已解压: WEB-INF/views/cluster/cluster.jsp
  已解压: WEB-INF/views/cluster/multicluster.jsp
  已解压: WEB-INF/views/topic/add_success.jsp
  已解压: WEB-INF/views/topic/manager.jsp
  已解压: WEB-INF/views/topic/mock.jsp
  已解压: WEB-INF/views/topic/add_failed.jsp
  已解压: WEB-INF/views/topic/msg.jsp
  已解压: WEB-INF/views/topic/topic_meta.jsp
  已解压: WEB-INF/views/topic/export.jsp
  已解压: WEB-INF/views/topic/create.jsp
  已解压: WEB-INF/views/topic/list.jsp
  已解压: WEB-INF/views/system/notice.jsp
  已解压: WEB-INF/views/system/resource.jsp
  已解压: WEB-INF/views/system/user.jsp
  已解压: WEB-INF/views/system/role.jsp
  已解压: WEB-INF/views/consumers/offset_realtime.jsp
  已解压: WEB-INF/views/consumers/offset_consumers.jsp
  已解压: WEB-INF/views/consumers/consumers.jsp
  已解压: WEB-INF/views/public/tcss.jsp
  已解压: WEB-INF/views/public/script.jsp
  已解压: WEB-INF/views/public/css.jsp
  已解压: WEB-INF/views/public/tagcss.jsp
  已解压: WEB-INF/views/public/navbar.jsp
  已解压: WEB-INF/views/public/kindeditor.jsp
  已解压: WEB-INF/views/public/tscript.jsp
  已解压: WEB-INF/views/alarm/history.jsp
  已解压: WEB-INF/views/alarm/add_success.jsp
  已解压: WEB-INF/views/alarm/rule_list.jsp
  已解压: WEB-INF/views/alarm/create_success.jsp
  已解压: WEB-INF/views/alarm/add_failed.jsp
  已解压: WEB-INF/views/alarm/add.jsp
  已解压: WEB-INF/views/alarm/create_failed.jsp
  已解压: WEB-INF/views/alarm/rule.jsp
  已解压: WEB-INF/views/alarm/modify.jsp
  已解压: WEB-INF/views/alarm/create.jsp
  已解压: WEB-INF/views/account/signin.jsp
  已解压: WEB-INF/views/main/index.jsp
  已解压: WEB-INF/views/error/405.jsp
  已解压: WEB-INF/views/error/404.jsp
  已解压: WEB-INF/views/error/403.jsp
  已解压: WEB-INF/views/error/503.jsp
  已解压: WEB-INF/views/error/500.jsp
  已解压: media/css/plugins/datatimepicker/daterangepicker.css
  已解压: media/css/plugins/select2/select2.min.css
  已解压: media/css/img/glyphicons-halflings.png
  已解压: media/css/public/bootstrap.min.css
  已解压: media/css/public/show-hint.css
  已解压: media/css/public/morris.css
  已解压: media/css/public/magicsuggest.css
  已解压: media/css/public/images/ui-icons_888888_256x240.png
  已解压: media/css/public/images/ui-bg_glass_75_ffffff_1x400.png
  已解压: media/css/public/images/ui-bg_glass_75_e6e6e6_1x400.png
  已解压: media/css/public/images/ui-icons_222222_256x240.png
  已解压: media/css/public/images/ui-bg_highlight-soft_75_cccccc_1x100.png
  已解压: media/css/public/images/ui-icons_454545_256x240.png
  已解压: media/css/public/codemirror.css
  已解压: media/css/public/bootstrap-treeview.min.css
  已解压: media/css/public/jquery.terminal.min.css
  已解压: media/css/public/bootstrap-tokenfield.css
  已解压: media/css/public/dataTables.bootstrap.min.css
  已解压: media/css/public/sb-admin.css
  已解压: media/css/public/font-awesome.min.css
  已解压: media/css/public/awesome-bootstrap-checkbox.css
  已解压: media/css/public/tokenfield-typeahead.css
  已解压: media/css/public/account/main.css
  已解压: media/css/public/account/hfd.ttf
  已解压: media/css/public/account/hfc.ttf
  已解压: media/css/fonts/glyphicons-halflings-regular.woff
  已解压: media/css/fonts/glyphicons-halflings-regular.woff2
  已解压: media/css/fonts/glyphicons-halflings-regular.ttf
  已解压: media/css/fonts/fontawesome-webfont.ttf
  已解压: media/css/fonts/fontawesome-webfont.woff
  已解压: media/js/plugins/datatimepicker/daterangepicker.js
  已解压: media/js/plugins/datatimepicker/moment.min.js
  已解压: media/js/plugins/codemirror/codemirror.js
  已解压: media/js/plugins/codemirror/show-hint.js
  已解压: media/js/plugins/codemirror/sql.js
  已解压: media/js/plugins/codemirror/sql-hint.js
  已解压: media/js/plugins/echart/macarons.js
  已解压: media/js/plugins/echart/echarts.min.js
  已解压: media/js/plugins/datatables/dataTables.bootstrap.min.js
  已解压: media/js/plugins/datatables/jquery.dataTables.min.js
  已解压: media/js/plugins/terminal/jquery.terminal.min.js
  已解压: media/js/plugins/d3/d3.layout.js
  已解压: media/js/plugins/d3/d3.js
  已解压: media/js/plugins/d3/d3.v3.min.js
  已解压: media/js/plugins/d3/liquidFillGauge.js
  已解压: media/js/plugins/tokenfield/bootstrap-tokenfield.js
  已解压: media/js/plugins/select2/select2.min.js
  已解压: media/js/plugins/select2/select2.min.js.bak
  已解压: media/js/plugins/magicsuggest/magicsuggest.js
  已解压: media/js/public/navbar.js
  已解压: media/js/public/bootstrap.min.js
  已解压: media/js/public/bootstrap-treeview.min.js
  已解压: media/js/public/jquery.js
  已解压: media/js/main/metrics/zk.js
  已解压: media/js/main/metrics/brokers.js
  已解压: media/js/main/metrics/kafka.js
  已解压: media/js/main/cluster/cluster.js
  已解压: media/js/main/cluster/zkcli.js
  已解压: media/js/main/cluster/multicluster.js
  已解压: media/js/main/index.js
  已解压: media/js/main/topic/export.js
  已解压: media/js/main/topic/create.js
  已解压: media/js/main/topic/list.js
  已解压: media/js/main/topic/msg.js
  已解压: media/js/main/topic/mock.js
  已解压: media/js/main/topic/manager.js
  已解压: media/js/main/topic/topic.meta.js
  已解压: media/js/main/system/notice.js
  已解压: media/js/main/system/role.js
  已解压: media/js/main/system/resource.js
  已解压: media/js/main/system/user.js
  已解压: media/js/main/alarm/create.js
  已解压: media/js/main/alarm/add.js
  已解压: media/js/main/alarm/history.js
  已解压: media/js/main/alarm/modify.js
  已解压: media/js/main/consumer/offset.consumer.js
  已解压: media/js/main/consumer/consumers.js
  已解压: media/js/main/consumer/offset.realtime.js
  已解压: media/js/main/account/signin.js
  已解压: media/js/main/error/error.js
  已解压: media/img/favicon.ico
  已解压: media/img/ke_login.png
  已创建: META-INF/maven/
  已创建: META-INF/maven/org.smartloli.kafka.eagle/
  已创建: META-INF/maven/org.smartloli.kafka.eagle/kafka-eagle-web/
  已解压: META-INF/maven/org.smartloli.kafka.eagle/kafka-eagle-web/pom.xml
  已解压: META-INF/maven/org.smartloli.kafka.eagle/kafka-eagle-web/pom.properties
*******************************************************************
* Kafka Eagle system monitor port successful... 
*******************************************************************
[2021-11-11 11:37:01] INFO: Status Code[0]
[2021-11-11 11:37:01] INFO: [Job done!]
Welcome to
    __ __    ___     ____    __ __    ___            ______    ___    ______    __     ______
   / //_/   /   |   / __/   / //_/   /   |          / ____/   /   |  / ____/   / /    / ____/
  / ,<     / /| |  / /_    / ,<     / /| |         / __/     / /| | / / __    / /    / __/   
 / /| |   / ___ | / __/   / /| |   / ___ |        / /___    / ___ |/ /_/ /   / /___ / /___   
/_/ |_|  /_/  |_|/_/     /_/ |_|  /_/  |_|       /_____/   /_/  |_|\____/   /_____//_____/   
                                                                                             

Version 1.3.7
*******************************************************************
* Kafka Eagle Service has started success.
* Welcome, Now you can visit 'http://192.168.7.102:8048/ke'
* Account:admin ,Password:123456
*******************************************************************
* <Usage> ke.sh [start|status|stop|restart|stats] </Usage>
* <Usage> https://www.kafka-eagle.org/ </Usage>
*******************************************************************

注意：启动之前需要先启动 ZK 以及 KAFKA

11.登录页面查看监控数据

牛逼太酷了

http://192.168.9.102:8048/ke

在这里插入图片描述

开启消费者

package org.example.consumer;

import org.apache.kafka.clients.consumer.ConsumerConfig;
import org.apache.kafka.clients.consumer.ConsumerRecord;
import org.apache.kafka.clients.consumer.ConsumerRecords;
import org.apache.kafka.clients.consumer.KafkaConsumer;

import java.util.Arrays;
import java.util.Properties;

/**
 * 操作Kafka-web 的代码
 * @ClassName MyConsumer4
 * @Author 小坏
 * @Date 2021/11/11、15:08
 * @Version 1.0
 */
public class MyConsumer4 {
    public static void main(String[] args) {
        Properties properties = new Properties();

        properties.put(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, "hadoop102:9092");
        //开启自动提交
        properties.put(ConsumerConfig.ENABLE_AUTO_COMMIT_CONFIG, true);
        //自动提交的延迟
        properties.put(ConsumerConfig.AUTO_COMMIT_INTERVAL_MS_CONFIG, "1000");

        //Key,Value 的反序列化
        properties.put(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, "org.apache.kafka.common.serialization.StringDeserializer");
        properties.put(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, "org.apache.kafka.common.serialization.StringDeserializer");

        /**
         * 如何重复消费主题的数据、
         * 1、换一个组
         * 2、设置重置的offset
         *  properties.put(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "earliest");
         *
         *  为什么从最大的消费？
         *  一个组过来消费的时候、相当于断开重新连接、然后会去返回一下以前的数据、重新连接了就不能返回了、找不到了、
         *  系统就要给你一个、既然没有了就告诉你从这个地方消费、给的时候就有 earliest、latest最大最小值、然后按照这个去消费、】
         *  发现没有数据的时候同样把这个写进去、然后就从最大的消费
         */
        properties.put(ConsumerConfig.GROUP_ID_CONFIG, "atguigu0408");

        //创建消费者
        KafkaConsumer<String, String> consumer = new KafkaConsumer<String, String>(properties);

        consumer.subscribe(Arrays.asList("test"));


        //获取数据
        while (true) {
            ConsumerRecords<String, String> consumerRecords = consumer.poll(100);

            //解析并打印consumerRecords
            for (ConsumerRecord<String, String> consumerRecord : consumerRecords) {
                System.out.println(consumerRecord.key() + "--" + consumerRecord.value());
            }

        }

    }
}

查看监控
在这里插入图片描述

小坏讲微服务

关注

1
点赞
踩
0

收藏

觉得还不错? 一键收藏
0
评论
大数据之 Kafka 监控完整使用 (第五章)

大数据之 Kafka 监控完整使用一、Kafka Eagle1.修改kafka 启动命令2.上传压缩包3.解压到本地4.进入刚才解压的目录5.将 kafka-eagle-web-1.3.7-bin.tar.gz 解压至/opt/module6.修改名称7.给启动文件执行权限8.修改配置文件9.添加环境变量10.启动11.登录页面查看监控数据一、Kafka Eagle1.修改kafka 启动命令修改 kafka-server-start.sh 命令中if [ "x$KAFKA_HEAP_OPTS"
复制链接

扫一扫