功能描述:fluentd(td-agent) 提取日志存储到elasticsearch 中
环境配置:elasticsearch 添加了shield 插件。配置了用户角色。
fluentd(td-agent)相关配置如下:
<match **>
type elasticsearch_dyn_index
hosts loghub-es-01.yr.local:9200,loghub-es-02.yr.local:9200,loghub-es-03.yr.local:9200
buffer_queue_limit 512
buffer_chunk_limit 16m
# NOTE! type_name must be consitent with what we set when creating new index in ES
type_name applog
index_prefix_key app-name
index_dateformat %Y-%m-%d
utc_index false
message_key msg
parse_msg_timestamp true
time_key time
num_threads 8
retry_wait 5s
max_retry_wait 600s
disable_retry_limit true
flush_interval 5s
user es_admin
password es_admin
</match>
日志信息报warn信息如下:
2016-04-06 09:38:19 +0800 [warn]: temp