logstash数据采集差8小时问题及解决(原创)

作者:star

  1. 软件版本
软件名称版本号
elasticsearch2.1.1
logstash2.3.1
  1. 问题解决办法
    解决方案1:
input {
    jdbc {
      jdbc_connection_string => "jdbc:oracle:thin:@10.35.4.97:1521:OMS1"
      jdbc_driver_library => "/home/yarn/logstash-2.3.1/lib/ojdbc14.jar"
      jdbc_driver_class => "Java::oracle.jdbc.driver.OracleDriver"
      jdbc_user => "ypt_jk"
      jdbc_password => "ypt_jk123"
      schedule => "11 18 25 11 *"
      statement => "SELECT '省调' dkflag,id, '' as loglevel, CONTENT AS datastr,'oms' AS sources,'自动化' AS type  ,'人工录入' as generatetype,'' AS host, to_char(RECORDDATE,'YYYY-MM-DD HH24:MI:SS') AS modifydate,RECORDUSER as RECORDUSER,recordyy AS remark FROM AUTO.T_AUTORUNRECORD"
    }
}
filter {
    date {
        match => [ "modifydate", "yyyy-MM-dd HH:mm:ss'GMT+8'"]
        target => "modifydate"
    }
}

output {
    elasticsearch {
        document_id => "%{id}"
        hosts => ["10.35.8.23:9200"]
        index => "test_log"
    }
    stdout {
        codec => json_lines
    }
}

解决方案2:

input {
    jdbc {
      jdbc_connection_string => "jdbc:oracle:thin:@10.35.4.97:1521:OMS1"
      jdbc_driver_library => "/home/yarn/logstash-2.3.1/lib/ojdbc14.jar"
      jdbc_driver_class => "Java::oracle.jdbc.driver.OracleDriver"
      jdbc_user => "ypt_jk"
      jdbc_password => "ypt_jk123"
      schedule => "54 10 25 11 *"
      statement => "SELECT '省调' dkflag,'' as loglevel,CONTENT   AS datastr,'oms' AS sources,'监控' AS type,'系统生成' as generatetype,'' AS host,to_char(OPERATETIME,'YYYY-MM-DD HH24:MI:SS') AS modifydate,RECORDPERSON as RECORDUSER,'' AS remark FROM DISPATCH.T_CONTROL_LOG_CONTENT"
    }
}
filter {
    date {
        match => [ "modifydate", "yyyy-MM-dd HH:mm:ss"]
        target => "modifydate"
        locale => "en"
        timezone => "+00:00"
        }

}

output {
    elasticsearch {
        hosts => ["10.35.8.23:9200"]
        index => "log_t_control_log_content"
        document_type => "oms_log"
    }
    stdout {
        codec => json_lines
    }
}

经测试:两种方案都可以很好的解决此问题

方案三:修改logstash源码

vi logstash-2.3.1\vendor\bundle\jruby\1.9\gems\logstash-core-event-2.3.1-java\lib\logstash\timestamp.rb

UTC = org.joda.time.DateTimeZone.forID(“UTC”)  
修改为:  
UTC = org.joda.time.DateTimeZone.getDefault()  



作者:雨中星辰0
链接:https://www.jianshu.com/p/4bb5470da4a9
来源:简书
简书著作权归作者所有,任何形式的转载都请联系作者获得授权并注明出处。

  • 1
    点赞
  • 1
    收藏
    觉得还不错? 一键收藏
  • 1
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论 1
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值