ElastAlert对ELK日志进行邮箱报警

阿里云1000元通用代金券点此领取

ElastAlert是针对ELK收集的日志进行报警的一个框架，类似的还有KAAE和elastic公司自己出品的Watcher，可以根据自己的需求选择。

ElastAlert目前支持的报警方式有email，command调用短信，Slack，Telegram等，因为微信可以绑定邮箱提醒，等于间接支持了微信。

github地址： https://github.com/Yelp/elastalert

安装过程中需要使用到python2.7

git clone https://github.com/Yelp/elastalert.git
python setup.py install

具体过程可以看官方的安装教程：http://elastalert.readthedocs.io

我这里的需求是ELK收集的日志在一定的时间里，如果没有达到指定的次数则报警
config.yaml

# This is the folder that contains the rule yaml files
# Any .yaml file will be loaded as a rule
rules_folder: pmc_rules  # 规则文件存放的文件夹名称
# How often ElastAlert will query Elasticsearch
# The unit can be anything from weeks to seconds
run_every:
  minutes: 1  # 每隔一分钟查询一次Elasticsearch
# ElastAlert will buffer results from the most recent
# period of time, in case some log sources are not in real time
buffer_time:
  minutes: 15  # 结果缓存15分钟
# The Elasticsearch hostname for metadata writeback
# Note that every rule can have its own Elasticsearch host
es_host: 10.x.x.x  # Elasticsearch的地址
# The Elasticsearch port
es_port: 9200  #  Elasticsearch端口
# The AWS region to use. Set this when using AWS-managed elasticsearch
#aws_region: us-east-1
# The AWS profile to use. Use this if you are using an aws-cli profile.
# See http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html
# for details
#profile: test
# Optional URL prefix for Elasticsearch
#es_url_prefix: elasticsearch
# Connect with TLS to Elasticsearch                                                                                                        
#use_ssl: True  # ES都在内网环境使用，外网无法访问，所以没有使用ssl和账号密码                                                                                                                           
                                                                                                                                           
# Verify TLS certificates                                                                                                                  
#verify_certs: True                                                                                                                        
                                                                                                                                           
# GET request with body is the default option for Elasticsearch.                                                                           
# If it fails for some reason, you can pass 'GET', 'POST' or 'source'.                                                                     
# See http://elasticsearch-py.readthedocs.io/en/master/connection.html?highlight=send_get_body_as#transport                                
# for details                                                                                                                              
#es_send_get_body_as: GET                                                                                                                  
                                                                                                                                           
# Option basic-auth username and password for Elasticsearch                                                                                
#es_username: someusername                                                                                                                 
#es_password: somepassword                                                                                                                 
                                                                                                                                           
# Use SSL authentication with client certificates client_cert must be                                                                      
# a pem file containing both cert and key for client                                                                                       
#verify_certs: True                                                                                                                        
#ca_certs: /path/to/cacert.pem                                                                                                             
#client_cert: /path/to/client_cert.pem                                                                                                     
#client_key: /path/to/client_key.key                                                                                                       
                                                                                                                                           
# The index on es_host which is used for metadata storage                                                                                  
# This can be a unmapped index, but it is recommended that you run                                                                         
# elastalert-create-index to set a mapping                                                                                                 
writeback_index: elastalert_status    # 在安装完成后执行命令  elastalert-create-index 创建的索引名称                                                                                                    
                                                                                                                                           
# If an alert fails for some reason, ElastAlert will retry                                                                                 
# sending the alert until this time period has elapsed                                                                                     
alert_time_limit:   # 如果报警失败，会在两天内重试                                                                                                                       
   days: 2

规则文件autoDispatchAdvanceJob.yaml，这个文件必须在config.yaml中rules_folder指定的文件夹下，可以有多个文件

# Alert when the rate of events exceeds a threshold

# (Optional)
# Elasticsearch host
es_host: 10.x.x.x  # Elasticsearch的地址

# (Optional)
# Elasticsearch port
es_port: 9200

# (Required)
# Rule name, must be unique
name: autoDispatchAdvanceJob Stop  #  规则名称，不能重复，邮件标题就是这个名字

# (Required)
# Type of alert.
# the frequency rule type alerts when num_events events occur with timeframe time
type: flatline  #  规则类型，详细说明http://elastalert.readthedocs.io
# (Required)
# Index to search, wildcard supported
index: pmc-timejob # Elasticsearch中的索引名称，需要报警的日志
# (Required, frequency specific)
# Alert when this many documents matching the query occur within a timeframe
#num_events: 50 
threshold: 1  # 35分钟内查询内容需要出现的次数
# (Required, frequency specific)                                                                                                           
# num_events must occur within this amount of time to trigger an alert                                                                     
timeframe:                                                                                                                                 
  minutes: 35  # 时间间隔                                                                                                                                                                                                                                                                       
# (Required)                                                                                                                               
# A list of Elasticsearch filters used for find events                                                                                     
# These filters are joined with AND and nested in a filtered query                                                                         
# For more info: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl.html                                      
filter:                                                                                                                                     
- query:                                                                                                                                   
    query_string:  #  需要日志出现的字符串                                                                                                                        
      query: "autoDispatchAdvanceJob"                                                                                                                                                                                                                                          
# (Required)                                                                                                                               
# The alert is use when a match is found                                                                                                   
alert:     #  告警方式，这里使用QQ邮箱                                                                                                                                
- "email"                                                                                                                                                                                                                                                                     
smtp_host: smtp.qq.com                                                                                                                     
smtp_port: 587                                                                                                                             
#用户认证文件，需要user和password两个属性                                                                                                  
smtp_auth_file: /ELK/elastalert/smtp—file.yaml  # 这个文件包含发件邮箱的账号密码                                                                                                                                                                                                               
email_reply_to: "123333321@qq.com"                                                                                                                                                                                                                                 
from_addr: "123333321@qq.com"
# (required, email specific)
# a list of email addresses to send alerts to
email:  #  可以有多个接收邮箱
- "123456789@qq.com"
- "987564321@qq.com"

smtp—file.yaml

#发送邮件的邮箱
user: "123333321@qq.com"
##不是邮箱密码，是设置的POP3密码
password: "sdffnddflcvdhbi"

启动服务
python -m elastalert.elastalert --config /ELK/elastalert/config.yaml --verbose