zabbix和prometheus通过webhook推送告警至钉钉

zabbix和prometheus通过webhook推送告警至钉钉

之前告警都是通过邮件推送，总感觉及时性不够，就找了找网上大佬们的方案，非原创，只记录一下配置的过程。。。
zabbix：http://www.linuxidc.com/Linux/2017-12/149953.htm
prometheus：https://github.com/timonwong/prometheus-webhook-dingtalk

zabbix

• 添加钉钉机器人，获取webhook
• 下载插件
• copy插件至zabbix告警脚本路径

[root@localhost etc]# grep -i  alert  zabbix_server.conf|  sed '/^#/d'
AlertScriptsPath=/data/zabbix/alertscripts

[root@localhost etc]# ll /data/zabbix/alertscripts
total 9608
-rwxr-xr-x 1 root   root   5828913 Apr 28  2017 golang-zabbix-robot-64

• zabbix添加media types

name:   dingding_ops
type:   scripts
script name:    golang-zabbix-robot-64
Script parameters:
        -webhook=钉钉机器人webhook URL 
        -msg={ALERT.MESSAGE}
        -url=http://1.1.1.10
        -log=/tmp/dingding.log

• zabbix添加actions

Name:   dingding_ops


Default subject:    故障告警：{TRIGGER.STATUS}: {TRIGGER.NAME}
Default message:    
        <?xml version="1.0" encoding="UTF-8" ?>
        <root>
          <from>{HOSTNAME1}</from>
          <time>{EVENT.DATE} {EVENT.TIME}</time>
          <level>{TRIGGER.SEVERITY}</level>
          <name>{TRIGGER.NAME}</name>
          <key>{TRIGGER.KEY1}</key>
          <value>{ITEM.VALUE}</value>
          <now>{ITEM.LASTVALUE}</now>
          <id>{EVENT.ID}</id>
          <ip>{HOST.IP}</ip>
          <age>{EVENT.AGE}</age>
          <status>{EVENT.STATUS}</status>
        <acknowledgement> {EVENT.ACK.STATUS} </acknowledgement>
        <acknowledgementhistory> {EVENT.ACK.HISTORY}</acknowledgementhistory>
        </root>


Recovery subject:       恢复通知：{TRIGGER.STATUS}: {TRIGGER.NAME}
Recovery message:       
        <?xml version="1.0" encoding="UTF-8" ?>
        <root>
          <from>{HOSTNAME1}</from>
          <time>{EVENT.DATE} {EVENT.TIME}</time>
          <level>{TRIGGER.SEVERITY}</level>
          <name>{TRIGGER.NAME}</name>
          <key>{TRIGGER.KEY1}</key>
          <value>{ITEM.VALUE}</value>
          <now>{ITEM.LASTVALUE}</now>
          <id>{EVENT.ID}</id>
          <ip>{HOST.IP}</ip>
          <color>FF4A934A</color>
          <age>{EVENT.AGE}</age>
          <recoveryTime>{EVENT.RECOVERY.DATE} {EVENT.RECOVERY.TIME}</recoveryTime>
          <status>{EVENT.RECOVERY.STATUS}</status>
        </root>

• user下添加media

type：       dingding_ops
send to:        随便填
其余默认

Prometheus

• 添加钉钉机器人，获取webhook
• 下载插件并编译（不会编译的也可以直接下载编译好的二进制文件）
• 编辑启动脚本（请替换为自己的webhook URL 及 ding.profile）

cat dingding_start.sh

nohup /usr/local/alertmanager-0.8.0.linux-amd64/prometheus-webhook-dingtalk --ding.profile="ops_dingding=https://oapi.dingtalk.com/robot/send?access_token=11111111111"  --ding.profile="dev_dingding=https://oapi.dingtalk.com/robot/send?access_token=222222222222" 2>&1 1>/usr/local/alertmanager-0.8.0.linux-amd64/dingding.log & 

• 编辑alertmanager.yml，增加web_hook配置并重启alertmanager

- name: "ops_dingding" 
  webhook_configs:
    - send_resolved: true 
      url: "http://localhost:8060/dingtalk/ops_dingding/send"
- name: "dev_dingding"
  webhook_configs:
    - send_resolved: true
      url: "http://localhost:8060/dingtalk/dev_dingding/send"