logstash利用ruby语言写复杂的处理逻辑

最新推荐文章于 2024-04-07 10:56:04 发布

Fang Shuzhi

最新推荐文章于 2024-04-07 10:56:04 发布

阅读量1.9w

点赞数 1

分类专栏：大数据可视化电商金融风控文章标签： ruby filter logstash 正则表达式

本文链接：https://blog.csdn.net/ty_0930/article/details/52609360

版权

大数据可视化同时被 2 个专栏收录

24 篇文章 0 订阅

订阅专栏

电商金融风控

9 篇文章 0 订阅

订阅专栏

在应用中，需要对logstash读取的数据进行字符判断、字符串截取等操作。同时也需要用到正则表达式对某些匹配字符进行替换。
直接上配置文件逻辑：

input{
  stdin{
     # codec=>rubydebug
    }
}
filter{
#   codec=>rubydebug
    mutate{
    split=>["message"," "]
        add_field => {
            "field1" => "%{[message][0]}"
        }   
        add_field => {
            "field2" => "%{[message][1]}"
        }
        remove_field => ["message"]
    }

    ruby{
        code=>"
        length = event['field1'].length
    #   event['field1'] = event['field1'][length-1,length]
        lst = event['field1'][length-1,length]
        if lst == '{'
            index = event['field1'].rindex(',')
            jsonStr = '{' + event['field1'][0,index]
            event['field1'] = jsonStr.gsub(/\s/, '')
        elsif lst == '}'
            index = event['field1'].rindex(']')
            jsonStr = '{' + event['field1'][0,index]
            event['field1'] = jsonStr.gsub(/\s/, '')
        end
        #   event['field1'] = event['field1'].rindex(',')
        #   event['field1'] = Time.parse(event['field1']).to_f*1000
        "
    }
#   json{
#       source => "field1"
#       target => "field1"
#   }
}
output{
  stdout{
    codec=>rubydebug
  }
}