使用grafana+prometheus监控flink任务

1,软件安装:

grafana+prometheus 安装教程请参考文章: 安装教程

2,flink配置

方案一: 修改文件conf/flink-conf.yaml

metrics.reporter.promgateway.class: org.apache.flink.metrics.prometheus.PrometheusPushGatewayReporter
#这里填写安装的prometheusGateway的ip
metrics.reporter.promgateway.host: xx.xx.xx.xx
metrics.reporter.promgateway.port: 9091
#此处填写jobName
metrics.reporter.promgateway.jobName: xxxx
metrics.reporter.promgateway.randomJobNameSuffix: true
metrics.reporter.promgateway.deleteOnShutdown: false
# 为自己的job添加可被区分或者查询的key, aBcDf 在下面会用到
metrics.reporter.promgateway.groupingKey: jobUqId=aBcDf 

方案二, flink on yarn直接在命令行上添加, 会更方便:

-yD metrics.reporter.promgateway.class=org.apache.flink.metrics.prometheus.PrometheusPushGatewayReporter \
-yD metrics.reporter.promgateway.host=xxx.xxx.xxx.xxx -yD metrics.reporter.promgateway.port=9091 \
-yD metrics.reporter.promgateway.jobName=flink-iot-anlz-log \
-yD metrics.reporter.promgateway.randomJobNameSuffix=true \
-yD metrics.reporter.promgateway.deleteOnShutdown=false \
-yD metrics.reporter.promgateway.groupingKey=jobUqId=aBcDf 

然后将任务启动.

2,监控查看