prometheus 监控 主机存活 监控 blackbox_exporter

blackbox_exporter是Prometheus 官方提供的 exporter 之一，可以提供 http、dns、tcp、icmp 的监控数据采集。

blackbox_exporter 应用场景

1. ICMP 测试
   主机探活机制

2. HTTP 测试
   定义 Request Header 信息
   判断 Http status / Http Respones Header / Http Body 内容

3. TCP 测试
   业务组件端口状态监听
   应用层协议定义与监听

4. POST 测试
   接口联通性

5. SSL 证书过期时间

6. 自定义测试（扩展）

下载

各个版本的blackbox_exporter https://github.com/prometheus/blackbox_exporter/releases

$ tar -xvf blackbox_exporter-0.19.0.linux-amd64.tar.gz
$ cd blackbox_exporter-0.19.0.linux-amd64
$ ./blackbox_exporter --version
blackbox_exporter, version 0.19.0 (branch: HEAD, revision: 5d575b88eb12c65720862e8ad2c5890ba33d1ed0)
  build user:       root@2b0258d5a55a
  build date:       20210510-12:56:44
  go version:       go1.16.4
  platform:         linux/amd64

配置开机启动

cat /usr/lib/systemd/system/blackbox_exporter.service 
[Unit]
Description=blackbox_exporter
After=network.target

[Service]
User=root
Type=simple
ExecStart=/data/blackbox_exporter-0.19.0.linux-amd64/blackbox_exporter --config.file=/data/blackbox_exporter-0.19.0.linux-amd64/blackbox.yml
Restart=on-failure

[Install]
WantedBy=multi-user.target

blackbox_exporter 配置文件

配置文件默认就可以，如果有特殊需求，再按需修改

cat blackbox.yml 
modules:
  http_2xx:
    prober: http
  http_post_2xx:
    prober: http
    http:
      method: POST
  tcp_connect:
    prober: tcp
  pop3s_banner:
    prober: tcp
    tcp:
      query_response:
      - expect: "^+OK"
      tls: true
      tls_config:
        insecure_skip_verify: false
  ssh_banner:
    prober: tcp
    tcp:
      query_response:
      - expect: "^SSH-2.0-"
      - send: "SSH-2.0-blackbox-ssh-check"
  irc_banner:
    prober: tcp
    tcp:
      query_response:
      - send: "NICK prober"
      - send: "USER prober prober prober :prober"
      - expect: "PING :([^ ]+)"
        send: "PONG ${1}"
      - expect: "^:[^ ]+ 001"
  icmp:
    prober: icmp

启动blackbox_exporter

[root@NEWCWS-AS03 data]# systemctl start blackbox_exporter.service 
[root@NEWCWS-AS03 data]# systemctl status blackbox_exporter.service 
● blackbox_exporter.service - blackbox_exporter
   Loaded: loaded (/usr/lib/systemd/system/blackbox_exporter.service; disabled; vendor preset: disabled)
   Active: active (running) since 四 2021-06-17 17:06:17 CST; 10s ago
 Main PID: 35945 (blackbox_export)
    Tasks: 9
   Memory: 4.1M
   CGroup: /system.slice/blackbox_exporter.service
           └─35945 /data/blackbox_exporter-0.19.0.linux-amd64/blackbox_exporter --config.file=/data/blackbox_exporter-0.19.0.linux-amd64/blackbox.yml

6月 17 17:06:17 NEWCWS-AS03 systemd[1]: Started blackbox_exporter.
6月 17 17:06:17 NEWCWS-AS03 blackbox_exporter[35945]: level=info ts=2021-06-17T09:06:17.475Z caller=main.go:224 msg="Starting blackbox_exporter" version="(version=0.19.0, branch=HEAD, revision=...90ba33d1ed0)"
6月 17 17:06:17 NEWCWS-AS03 blackbox_exporter[35945]: level=info ts=2021-06-17T09:06:17.475Z caller=main.go:225 build_context="(go=go1.16.4, user=root@2b0258d5a55a, date=20210510-12:56:44)"
6月 17 17:06:17 NEWCWS-AS03 blackbox_exporter[35945]: level=info ts=2021-06-17T09:06:17.476Z caller=main.go:237 msg="Loaded config file"
6月 17 17:06:17 NEWCWS-AS03 blackbox_exporter[35945]: level=info ts=2021-06-17T09:06:17.476Z caller=main.go:385 msg="Listening on address" address=:9115
6月 17 17:06:17 NEWCWS-AS03 blackbox_exporter[35945]: level=info ts=2021-06-17T09:06:17.476Z caller=tls_config.go:191 msg="TLS is disabled." http2=false
Hint: Some lines were ellipsized, use -l to show in full.

在Prometheus上添加blackbox_exporter的配置

  - job_name: '主机网络监测'
    metrics_path: /probe
    params:
      module: [icmp] #对应blackbox_exporter.yml的配置
    file_sd_configs:
    - refresh_interval: 10s
      files:
      - "/data/prometheus-2.22.1.linux-amd64/conf.d/icmp_status.yml"
    relabel_configs:
      - source_labels: [__address__]
        target_label: __param_target
      - target_label: __address__
        replacement: 127.0.0.1:9115
      - source_labels: [__param_target]
        target_label: instance

cat conf.d/icmp_status.yml
- targets: ['192.168.65.220','192.168.65.171','1.1.1.1']
  labels:
    group: 'group1-网络PING监控'
- targets: ['192.168.80.198','192.168.80.199']
  labels:
    group: 'group2-网络PING监控'
- targets: ['192.168.80.142','192.168.80.143']
  labels:
    group: 'group3-网络PING监控' 

重新加载prometheus配置

测试配置是否正确

$./promtool check config prometheus.yml
Checking prometheus.yml
  SUCCESS: 0 rule files found

重新加载Prometheus配置

$ curl -X POST http://127.0.0.1:9090/-/reload (启用了--web.enable-lifecycle选项)

查看加入的监控信息

访问 http://127.0.0.1:9090/targets

Grafana中加入 blackbox_exporter监控数据

导入blackbox_exporter模板

此模板为9965号模板，数据源选择Prometheus 模板下载地址
https://grafana.com/grafana/dashboards/9965

注意

此模板需要安装饼状图插件 下载地址 https://grafana.com/grafana/plugins/grafana-piechart-panel
安装插件，重启grafana生效。

$ grafana-cli plugins install grafana-piechart-panel
$ service grafana-server restart

访问Grafana

CSDN_码404：prometheus 监控 主机存活 监控 blackbox_exporter