prometheus 监控 主机存活 监控 blackbox_exporter
blackbox_exporter是Prometheus 官方提供的 exporter 之一,可以提供 http、dns、tcp、icmp 的监控数据采集。
blackbox_exporter 应用场景
-
ICMP 测试
主机探活机制 -
HTTP 测试
定义 Request Header 信息
判断 Http status / Http Respones Header / Http Body 内容 -
TCP 测试
业务组件端口状态监听
应用层协议定义与监听 -
POST 测试
接口联通性 -
SSL 证书过期时间
-
自定义测试(扩展)
下载
各个版本的blackbox_exporter https://github.com/prometheus/blackbox_exporter/releases
$ tar -xvf blackbox_exporter-0.19.0.linux-amd64.tar.gz
$ cd blackbox_exporter-0.19.0.linux-amd64
$ ./blackbox_exporter --version
blackbox_exporter, version 0.19.0 (branch: HEAD, revision: 5d575b88eb12c65720862e8ad2c5890ba33d1ed0)
build user: root@2b0258d5a55a
build date: 20210510-12:56:44
go version: go1.16.4
platform: linux/amd64
配置开机启动
cat /usr/lib/systemd/system/blackbox_exporter.service
[Unit]
Description=blackbox_exporter
After=network.target
[Service]
User=root
Type=simple
ExecStart=/data/blackbox_exporter-0.19.0.linux-amd64/blackbox_exporter --config.file=/data/blackbox_exporter-0.19.0.linux-amd64/blackbox.yml
Restart=on-failure
[Install]
WantedBy=multi-user.target
blackbox_exporter 配置文件
配置文件默认就可以,如果有特殊需求,再按需修改
cat blackbox.yml
modules:
http_2xx:
prober: http
http_post_2xx:
prober: http
http:
method: POST
tcp_connect:
prober: tcp
pop3s_banner:
prober: tcp
tcp:
query_response:
- expect: "^+OK"
tls: true
tls_config:
insecure_skip_verify: false
ssh_banner:
prober: tcp
tcp:
query_response:
- expect: "^SSH-2.0-"
- send: "SSH-2.0-blackbox-ssh-check"
irc_banner:
prober: tcp
tcp:
query_response:
- send: "NICK prober"
- send: "USER prober prober prober :prober"
- expect: "PING :([^ ]+)"
send: "PONG ${1}"
- expect: "^:[^ ]+ 001"
icmp:
prober: icmp
启动blackbox_exporter
[root@NEWCWS-AS03 data]# systemctl start blackbox_exporter.service
[root@NEWCWS-AS03 data]# systemctl status blackbox_exporter.service
● blackbox_exporter.service - blackbox_exporter
Loaded: loaded (/usr/lib/systemd/system/blackbox_exporter.service; disabled; vendor preset: disabled)
Active: active (running) since 四 2021-06-17 17:06:17 CST; 10s ago
Main PID: 35945 (blackbox_export)
Tasks: 9
Memory: 4.1M
CGroup: /system.slice/blackbox_exporter.service
└─35945 /data/blackbox_exporter-0.19.0.linux-amd64/blackbox_exporter --config.file=/data/blackbox_exporter-0.19.0.linux-amd64/blackbox.yml
6月 17 17:06:17 NEWCWS-AS03 systemd[1]: Started blackbox_exporter.
6月 17 17:06:17 NEWCWS-AS03 blackbox_exporter[35945]: level=info ts=2021-06-17T09:06:17.475Z caller=main.go:224 msg="Starting blackbox_exporter" version="(version=0.19.0, branch=HEAD, revision=...90ba33d1ed0)"
6月 17 17:06:17 NEWCWS-AS03 blackbox_exporter[35945]: level=info ts=2021-06-17T09:06:17.475Z caller=main.go:225 build_context="(go=go1.16.4, user=root@2b0258d5a55a, date=20210510-12:56:44)"
6月 17 17:06:17 NEWCWS-AS03 blackbox_exporter[35945]: level=info ts=2021-06-17T09:06:17.476Z caller=main.go:237 msg="Loaded config file"
6月 17 17:06:17 NEWCWS-AS03 blackbox_exporter[35945]: level=info ts=2021-06-17T09:06:17.476Z caller=main.go:385 msg="Listening on address" address=:9115
6月 17 17:06:17 NEWCWS-AS03 blackbox_exporter[35945]: level=info ts=2021-06-17T09:06:17.476Z caller=tls_config.go:191 msg="TLS is disabled." http2=false
Hint: Some lines were ellipsized, use -l to show in full.
在Prometheus上添加blackbox_exporter的配置
- job_name: '主机网络监测'
metrics_path: /probe
params:
module: [icmp] #对应blackbox_exporter.yml的配置
file_sd_configs:
- refresh_interval: 10s
files:
- "/data/prometheus-2.22.1.linux-amd64/conf.d/icmp_status.yml"
relabel_configs:
- source_labels: [__address__]
target_label: __param_target
- target_label: __address__
replacement: 127.0.0.1:9115
- source_labels: [__param_target]
target_label: instance
cat conf.d/icmp_status.yml
- targets: ['192.168.65.220','192.168.65.171','1.1.1.1']
labels:
group: 'group1-网络PING监控'
- targets: ['192.168.80.198','192.168.80.199']
labels:
group: 'group2-网络PING监控'
- targets: ['192.168.80.142','192.168.80.143']
labels:
group: 'group3-网络PING监控'
重新加载prometheus配置
测试配置是否正确
$./promtool check config prometheus.yml
Checking prometheus.yml
SUCCESS: 0 rule files found
重新加载Prometheus配置
$ curl -X POST http://127.0.0.1:9090/-/reload (启用了--web.enable-lifecycle选项)
查看加入的监控信息
访问 http://127.0.0.1:9090/targets
Grafana中加入 blackbox_exporter监控数据
导入blackbox_exporter模板
此模板为9965号模板,数据源选择Prometheus 模板下载地址
https://grafana.com/grafana/dashboards/9965
注意
此模板需要安装饼状图插件 下载地址 https://grafana.com/grafana/plugins/grafana-piechart-panel
安装插件,重启grafana生效。
$ grafana-cli plugins install grafana-piechart-panel
$ service grafana-server restart