Logstash 实践之Redis日志解析

最新推荐文章于 2024-07-06 03:33:15 发布

joeywen

最新推荐文章于 2024-07-06 03:33:15 发布

阅读量4.8k

点赞数

分类专栏： Logstash 搜索引擎(ELK/Lucene/Solr) 文章标签： logstash redis

本文链接：https://blog.csdn.net/wzhg0508/article/details/50616495

版权

本文介绍了如何使用Logstash进行Redis集群日志的解析，包括logstash配置脚本和样例日志的解析输出，重点展示了Grok正则表达式的应用。

摘要由CSDN通过智能技术生成

Logstash 实践之Redis日志解析

logstash config 脚本配置

input {
    file {
        path => ["/apps/svr/logstash/log/redis1.log"]
        start_position => "beginning"
        type => "redis_cluster"
    }
}

filter {
    multiline {
        what => next
        pattern => "^(?!(\d)+).*$"
    }
    grok {
        match => ["message", "(?<pid>.\d+?):(?<role>\w?)\s+(?<log_time>%{MONTHDAY}\s+%{MONTH}\s+%{HOUR}:%{MINUTE}:%{SECOND}?)\s+(?<log_level>.?)\s%{GREEDYDATA:message}"]
        overwrite => ["message"]
    }

    if [log_level] == "*" {
        mutate{ update => {
  "log_level" => "NOTICE"}}
    }

    if [log_level] == "#" {
        mutate{ update => {
  "log_level" => "WARNING"}}
    }

    if [log_level] == "-" {
        mutate{ update => {
  "log_level" => "VERBOSE"}}
    }

    if [log_level] == "." {
        mutate{ update => {
  "log_level" => "DEBUG"}}
    }
}
output {
    stdout {
        codec => rubydebug
    }

}

grok的正确性可以在该网站检验Grok Test

Sample redis cluster log

230186:M 07 Jan 14:10:31.137 # Bad message length or signature received from Cluster bus.
230186:M 07 Jan 14:14:10.291 * FAIL message received from a36530f8df368550c186d9d8a2c5e39d3afe9b65 about 3d8bdc2e000031eb1f30f885d8a58fed4be270ed
230186:M 07 Jan 14:14:45.131 * Clear FAIL state for node 3d8bdc2e000031eb1f30f885d8a58fed4be270ed: is reachable again and nobody is serving its slots after some time.
230186:M 07 Jan 14:15:27.525 # Bad message length or signature received from Cluster bus.
230186:M 07 Jan 14:15:27.525 # Bad message length or signature received from Cluster bus.
230186:M <