2020-08-09

elastAlert告警

安装环境

• CentOS：7.4
• Python：3.6.9(官方文档为2.7，亲测3.6)
• pip：19.3
• elastalert：0.2.1
• elk：7.6.0

安装

1. 进入安装目录/usr/share，直接使用pip安装

pip  install  elastalert

2. 进入elastalert项目根目录执行安装插件命令

pip install "setuptools>=11.3"

python setup.py install

pip install "elasticsearch>=5.0.0"

3. 复制配置文件并修改相关配置

cp  config.yaml.example   config.yaml

vi config.yaml

设置es_host、es_host指向你的Elasticsearch，然后保存退出。

rules_folder: es_rules

# How often ElastAlert will query Elasticsearch
# The unit can be anything from weeks to seconds
run_every:
  seconds: 30

# ElastAlert will buffer results from the most recent
# period of time, in case some log sources are not in real time
buffer_time:
  seconds: 30

# The Elasticsearch hostname for metadata writeback
# Note that every rule can have its own Elasticsearch host
es_host: localhost

# The Elasticsearch port
es_port: 9200

# The AWS region to use. Set this when using AWS-managed elasticsearch
#aws_region: us-east-1

# The AWS profile to use. Use this if you are using an aws-cli profile.
# See http://docs.aws.amazon.com/cli/latest/userguide/cli-chap-getting-started.html
# for details
#profile: test

# Optional URL prefix for Elasticsearch
#es_url_prefix: elasticsearch

# Connect with TLS to Elasticsearch
use_ssl: false

# Verify TLS certificates
verify_certs: false

# GET request with body is the default option for Elasticsearch.
# If it fails for some reason, you can pass 'GET', 'POST' or 'source'.
# See http://elasticsearch-py.readthedocs.io/en/master/connection.html?highlight=send_get_body_as#transport
# for details
#es_send_get_body_as: GET

# Option basic-auth username and password for Elasticsearch
es_username: elastic
es_password: 1qaz@WSX

# Use SSL authentication with client certificates client_cert must be
# a pem file containing both cert and key for client
#verify_certs: True
#ca_certs: /path/to/cacert.pem
#client_cert: /path/to/client_cert.pem
#client_key: /path/to/client_key.key

# The index on es_host which is used for metadata storage
# This can be a unmapped index, but it is recommended that you run
# elastalert-create-index to set a mapping
writeback_index: elastalert_status
#writeback_alias: elastalert_alerts

# If an alert fails for some reason, ElastAlert will retry
# sending the alert until this time period has elapsed
alert_time_limit:
  days: 2

# Custom logging configuration
# If you want to setup your own logging configuration to log into
# files as well or to Logstash and/or modify log levels, use
# the configuration below and adjust to your needs.
# Note: if you run ElastAlert with --verbose/--debug, the log level of
# the "elastalert" logger is changed to INFO, if not already INFO/DEBUG.
#logging:
#  version: 1
#  incremental: false
#  disable_existing_loggers: false
#  formatters:
#    logline:
#      format: '%(asctime)s %(levelname)+8s %(name)+20s %(message)s'
#
#    handlers:
#      console:
#        class: logging.StreamHandler
#        formatter: logline
#        level: DEBUG
#        stream: ext://sys.stderr
#
#      file:
#        class : logging.FileHandler
#        formatter: logline
#        level: DEBUG
#        filename: elastalert.log
#
#    loggers:
#      elastalert:
#        level: WARN
#        handlers: []
#        propagate: true
#
#      elasticsearch:
#        level: WARN
#        handlers: []
#        propagate: true
#
#      elasticsearch.trace:
#        level: WARN
#        handlers: []
#        propagate: true
#
#      '':  # root logger
#        level: WARN
#          handlers:
#            - console
#            - file
#        propagate: false

配置文件解析

此处的配置文件是指config.yaml文件。
rules_folder：ElastAlert将加载规则配置文件的地方，它将尝试加载文件夹中的每个.yaml文件。
run_every：ElastAlert查询Elasticsearch的频率。
buffer_time：是查询窗口的大小，从每个查询运行的时间向后延伸。对于其中use_count_query或use_terms_query设置为true的规则，此值将被忽略。
es_host：是Elasticsearch集群的地址，ElastAlert将存储有关其状态、查询运行、警报和错误的数据。每个规则也可以设置不同的elasticsearch主机进行查询。
es_port：Elasticsearch对应的端口。
use_ssl： （可选的）是否使用TLS;连接到es_host；设置为True或False。
verify_certs： （可选的）是否验证TLS证书; 设置为True或False，默认是True。
client_cert： （可选的）PEM证书的路径。
client_key： （可选的） 作为客户端密钥使用的私钥文件的路径。
ca_certs： （可选的） 用于验证SSL连接的CA证书的路径。
es_username： （可选的） 用于连接Elasticsearch的basic-auth用户名。
es_password： （可选的） 用于连接Elasticsearch的密码。
es_url_prefix： （可选的） Elasticsearch端点的URL前缀。
es_send_get_body_as： （可选的） 查询Elasticsearch方法- GET，POST或source，默认是GET。
writeback_index：是ElastAlert将存储数据的索引名称。
alert_time_limit： 是失败警报的重试窗口。

创建es索引

elastalert-create-index这个命令会在elasticsearch创建索引，便于ElastAlert将有关其查询及其警报的信息和元数据保存回Elasticsearch。这不是必须的步骤，但是强烈建议创建。因为对于审计，测试很有用，并且重启elastalert不影响计数和发送alert。默认情况下，创建的索引叫elastalert_status。

elastalert-create-index  --host 127.0.0.1 --port 9200 --username elastic  --password 1qaz@WSX --no-ssl --no-verify-certs

配置smtp

在/usr/share/elastalert目录下创建文件smtp_auth_file.yaml，内容如下：

#邮箱用户名
user: example@email.com

#不是邮箱密码，是设置的SMTP密码
password: XXXXX

6. 在es_rules目录下创建配置文件frequency.yaml：

# Alert when the rate of events exceeds a threshold

name: cdkey rule
#writeback_index: elastalert_status

type: frequency

index: elk-yunchang-cdkey-*

num_events: 1

timeframe:
  minutes: 1

filter:
- query:
     query_string:
       query: "Pool empty"

smtp_host: smtp.qq.com
smtp_port: 587
smtp_auth_file: /usr/share/elastalert/smtp_auth_file.yaml
email_reply_to: 939496422@qq.com
from_addr: 939496422@qq.com
alert:
- "email"

# (required, email specific)
# a list of email addresses to send alerts to
email:
- "weihang@game2sky.com"
[likenan@VM_51_58_centos es_rules]$ ls
cdkey.yaml  frequency.yaml
[likenan@VM_51_58_centos es_rules]$ cat frequency.yaml
# Alert when the rate of events exceeds a threshold

# (Optional)
# Elasticsearch host


# (Optional)
# Elasticsearch port
#es_port: 9200

# (OptionaL) Connect with SSL to Elasticsearch
#use_ssl:false
#verify_certs: false
# (Optional) basic-auth username and password for Elasticsearch
#es_username: elastic
#es_password: 888888

# (Required)
# Rule name, must be unique
name: frequency rule
#writeback_index: elastalert_status
# (Required)
# Type of alert.
# the frequency rule type alerts when num_events events occur with timeframe time
type: frequency

# (Required)
# Index to search, wildcard supported
index: test-2020

# (Required, frequency specific)
# Alert when this many documents matching the query occur within a timeframe
num_events: 1

# (Required, frequency specific)
# num_events must occur within this amount of time to trigger an alert
timeframe:
  minutes: 1

# (Required)
# A list of Elasticsearch filters used for find events
# These filters are joined with AND and nested in a filtered query
# For more info: http://www.elasticsearch.org/guide/en/elasticsearch/reference/current/query-dsl.html
filter:
- query:
     query_string:
       query: "field: value"

# (Required)
# The alert is use when a match is found
smtp_host: smtp.qq.com
smtp_port: 587
smtp_auth_file: /usr/share/elastalert/smtp_auth_file.yaml
email_reply_to: 939496433@qq.com
from_addr: 939496433@qq.com
alert:
- "email"

# (required, email specific)
# a list of email addresses to send alerts to
email:
- "939496434@qq.com"

上述规则表示：在elastalert执行的一分钟内，出现一条有field值为value日志，则触发告警，并且告警通知将以email的形式从939496433@qq.com邮箱发送给939496434@qq.com

运行

重新加载一个新规则

cd /var/share/elastalert
python -m elastalert.elastalert --verbose --rule es_rules/frequency.yaml

8. 给Elasticsearch添加测试数据

POST /test-2020/test
{
  "@timestamp": "2020-07-03T08:27:00.000Z",
 "field":"value is 1"
}

注意：上述的@timestamp是UTC时间，也就是说这条数据是在2020-07-03 16:27:00产生的。

可在邮箱中看到如下图所示的邮件,收到邮件中会包含当前符合告警规则的日志数据。