1、下载
https://prometheus.io/download/
2、解压到/usr/local/prometheus/,并启动
tar -zxvf alertmanager-0.27.0.linux-amd64.tar.gz -C /usr/local/prometheus/
#重命名
cd /usr/local/prometheus/
mv alertmanager-0.27.0.linux-amd64 alertmanager-0.27.0
#启动
cd /usr/local/prometheus/alertmanager-0.27.0
nohup /usr/local/prometheus/alertmanager-0.27.0/alertmanager --config.file=alertmanager.yml &
3、访问
http://ip:9093
4、设置企业微信群机器人告警
需要下载一个开源软件
下载地址:https://gitee.com/linge365/alertmanager-wechatrobot-webhook
官方安装过程如下:
mkdir /opt/prometheus
cd /opt/prometheus
git clone https://gitee.com/linge365/alertmanager-wechatrobot-webhook.git
cd alertmanager-wechatrobot-webhook
mv alertmanager-wechatrobot-webhook.service /etc/systemd/system/
useradd -M -s /usr/sbin/nologin prometheus
chown -R prometheu.prometheus /opt/prometheus
启动:
systemctl start alertmanager-wechatrobot-webhook
#默认端口8999,查看是否启动
lsof -i:8999
5、Prometheus集成机器人告警组件
1、修改Prometheus配置文件(/usr/local/prometheus/prometheus-2.54.0/prometheus.yml)
# Alertmanager configuration
alerting:
alertmanagers:
- static_configs:
- targets:
- 'localhost:9093'
使配置文件生效
#验证文件是否格式正确
./promtool check config prometheus.yml
#加载
curl -X POST http://ip:9090/-/reload
6、配置机器人
修改配置文件:/usr/local/prometheus/alertmanager-0.27.0/alertmanager.yml
global:
resolve_timeout: 5m
route:
group_by: ['alertname']
group_wait: 30s
group_interval: 5m
repeat_interval: 4h
receiver: 'wechat'
receivers:
- name: "wechat"
webhook_configs:
- url: 'http://你自己的机器ip:8999/webhook?key=你自己的机器人key'
send_resolved: true
7、测试告警
这里又一个告警脚本node.sh,告警磁盘使用率超过80%,测试时可以设置小一点,放到服务器上,并配置Prometheus配置文件
node.sh:
groups:
- name: node.rules
rules:
- alert: NodeFilessystemUsage
expr: 100 - (node_filesystem_free_bytes{fstype=~"ext4|xfs",mountpoint="/"} / node_filesystem_size_bytes{fstype=~"ext4|xfs",mountpoint="/"}
*100) > 80
for: 1m
labels:
serverity: warning
annotations:
summary: "主机 {{ $labels.instance }} : {{ $labels.mountpoint }} 磁盘使用率过高"
description: "{{ $labels.instance }} : {{ $labels.mountpoint }} 磁盘使用率超过80% (当前值: {{ $value }}) "
配置Prometheus配置文件(/usr/local/prometheus/prometheus-2.54.0/prometheus.yml)
alerting:
alertmanagers:
- static_configs:
- targets:
- '你的alert机器ip:9093'
rule_files:
- "你的node.yml文件路径"
8、设置告警模板(可以不管,系统有默认模板)
1、在alertmanager目录下创建企业微信配置文件,设置消息模板,wechat.tmpl
{{ define "wechat.default.message" }}
{{ range $i, $alert :=.Alerts }}
========监控报警==========
告警状态:{{ .Status }}
告警级别:{{ $alert.Labels.severity }}
告警类型:{{ $alert.Labels.alertname }}
告警应用:{{ $alert.Annotations.summary }}
告警主机:{{ $alert.Labels.instance }}
告警详情:{{ $alert.Annotations.description }}
触发阀值:{{ $alert.Annotations.value }}
告警时间:{{ $alert.StartsAt.Format "2006-01-02 15:04:05" }}
========end=============
{{ end }}
{{ end }}
2、修改alertmanager配置文件(/usr/local/prometheus/alertmanager-0.27.0/alertmanager.yml)
#templates:
- 'wechat.tmpl'