Prometheus告警推送蓝信webhook机器人

Forced O&M personnel

已于 2024-03-07 13:39:58 修改

阅读量630

点赞数 8

文章标签：机器人

于 2024-01-04 18:02:00 首次发布

本文链接：https://blog.csdn.net/weixin_49566876/article/details/135392327

版权

本文介绍了如何通过Python脚本实现与alertmanager的交互，接收告警信息，解析并重构消息，最后通过蓝信webhook发送告警通知。脚本被封装到Docker容器中，并配合DockerCompose进行自动化部署和管理。

摘要由CSDN通过智能技术生成

实现逻辑

通过Python脚本启动一个http接口用来接收alertmanager推送的告警信息，通过解析接口信息用来截取关键的告警信息，重构告警消息体，并按照蓝信webhook的接口格式进行推送。
为了方便管理启停，并防止脚本防止后台被误停止，将python脚本封装为一个容器进行管理。

告警消息处理流程

prometheus触发告警规则，发送告警信息给alertmanager，alertmanager接收到告警信息发送给python脚本的webhook接口，python脚本接收到告警信息后再转发给蓝信的webhook地址。

具体实现步骤

编写脚本【测试机IP: 192.168.0.161】

vi alert.py

import requests
import json
import time
import logging

from flask import Flask, request

#日志设置
logging.basicConfig(filename='alert.log', level=logging.DEBUG, format='%(asctime)s - %(levelname)s - %(message)s')
logging.debug('Debugging information')
logging.info('Informational message')
logging.warning('Warning: Proceed with caution')
logging.error('Error occurred')
logging.critical('Critical error -- cannot continue')

app = Flask(__name__)

timestamp = int(time.time())

@app.route('/webhook', methods=['POST'])
def webhook():
    #解析消息体
    data = request.get_json()
    logging.debug(f'接收消息体为：{data}')
    #拆分消息体
    alert = data['alerts'][0]
    alert_name = alert['labels']['alertname']
    alert_value = alert['annotations']['value']
    alert_message = alert['annotations']['summary']
    #重构消息体
    alerting = f"告警名称: {alert_name}\n当前值:{alert_value}\n告警内容:{alert_message}"

    payload = {
        #"sign": "8441D80B5A6664B38029D55EA9D65D8D",
        #"timestamp": timestamp,
        "msgType": "text",
        "msgData": {
            "text": {
                "content": alerting
                }
             }
           }

    webhook_url = '蓝信webhook-url'
    headers = {'Content-Type': 'application/json'}
    #执行http方法给webhook地址发送消息
    response = requests.post(webhook_url, headers=headers, data=json.dumps(payload))
    #打印日志
    logging.debug(f'推送消息内容：{alerting}')
    logging.debug(f'请求URL: {response.request.url}')
    logging.debug(f'请求头: {response.request.headers}')
    logging.debug(f'请求体: {response.request.body}')
    logging.debug(f'请求结果: {response.text}')

    if response.status_code == 200:
        return 'Alert forwarded successfully'
    else:
        return 'Failed to forward alert'
#设置程序启动监听IP+PORT
if __name__ == '__main__':
    app.run(host='0.0.0.0',port=5000)

构建docker镜像
编写dockerfile

vim Dockerfile

from centos:7
MAINTAINER yw <xxxxx@95114.cn>
RUN yum -y install python3&&mkdir -p /app/logs&&pip3 install requests flask datetime
ENV LANG="en_US.UTF-8"
COPY alert.py /app
WORKDIR /app
CMD python3 /app/alert.py

构建镜像

docker build -t webhook-lanxin:latest .

编写yaml文件

vi docker-compose-lanxin.yml

version: "3"
services:
  lanxin-alert:
    container_name: "lanxin-alert"
    image: webhook-lanxin:latest
    restart: always
    user: root
    ports:
      - "5000:5000"
    volumes:
      - ./logs:/app/logs

启动脚本

docker-compose -f docker-compose-lanxin.yml up -d

alertmanager配置

vim alertmanager.yml

  - name: 'webhook'
    email_configs:
    - to: 'test@95114.cn'
      html: '{{ template "mail.html" . }}'
      headers: { Subject: "[WARN] 报警邮件-webhook测试"}
      send_resolved: true
    webhook_configs:
    - url: 'http://192.168.0.161:5000/webhook'   #配置为刚启动的python脚本的IP+Port