flume java.lang.IllegalArgumentException: Timestamp must be positive

最新推荐文章于 2024-04-30 15:42:44 发布

风少年~

最新推荐文章于 2024-04-30 15:42:44 发布

阅读量734

点赞数

分类专栏：大数据遇见的异常

本文链接：https://blog.csdn.net/albg_boy/article/details/104611803

版权

遇见的异常同时被 2 个专栏收录

25 篇文章 0 订阅

订阅专栏

大数据

23 篇文章 1 订阅

订阅专栏

报错：

java.lang.IllegalArgumentException: Timestamp must be positive
        at com.google.common.base.Preconditions.checkArgument(Preconditions.java:88)
        at org.apache.flume.tools.TimestampRoundDownUtil.roundDownField(TimestampRoundDownUtil.java:88)
        at org.apache.flume.tools.TimestampRoundDownUtil.roundDownTimeStampMinutes(TimestampRoundDownUtil.java:58)
        at org.apache.flume.formatter.output.BucketPath.roundDown(BucketPath.java:346)
        at org.apache.flume.formatter.output.BucketPath.replaceShorthand(BucketPath.java:242)
        at org.apache.flume.formatter.output.BucketPath.escapeString(BucketPath.java:432)
        at org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:380)
        at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
        at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
        at java.lang.Thread.run(Thread.java:745)
02 Mar 2020 14:42:19,819 ERROR [SinkRunner-PollingRunner-DefaultSinkProcessor] (org.apache.flume.SinkRunner$PollingRunner.run:160)  - Unable to deliver event. Exception follows.
org.apache.flume.EventDeliveryException: java.lang.IllegalArgumentException: Timestamp must be positive
        at org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:463)
        at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
        at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
        at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.IllegalArgumentException: Timestamp must be positive
        at com.google.common.base.Preconditions.checkArgument(Preconditions.java:88)
        at org.apache.flume.tools.TimestampRoundDownUtil.roundDownField(TimestampRoundDownUtil.java:88)
        at org.apache.flume.tools.TimestampRoundDownUtil.roundDownTimeStampMinutes(TimestampRoundDownUtil.java:58)
        at org.apache.flume.formatter.output.BucketPath.roundDown(BucketPath.java:346)
        at org.apache.flume.formatter.output.BucketPath.replaceShorthand(BucketPath.java:242)
        at org.apache.flume.formatter.output.BucketPath.escapeString(BucketPath.java:432)
        at org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:380)
        ... 3 more
02 Mar 2020 14:42:24,821 ERROR [SinkRunner-PollingRunner-DefaultSinkProcessor] (org.apache.flume.sink.hdfs.HDFSEventSink.process:459)  - process failed
java.lang.IllegalArgumentException: Timestamp must be positive
        at com.google.common.base.Preconditions.checkArgument(Preconditions.java:88)
        at org.apache.flume.tools.TimestampRoundDownUtil.roundDownField(TimestampRoundDownUtil.java:88)
        at org.apache.flume.tools.TimestampRoundDownUtil.roundDownTimeStampMinutes(TimestampRoundDownUtil.java:58)
        at org.apache.flume.formatter.output.BucketPath.roundDown(BucketPath.java:346)
        at org.apache.flume.formatter.output.BucketPath.replaceShorthand(BucketPath.java:242)
        at org.apache.flume.formatter.output.BucketPath.escapeString(BucketPath.java:432)
        at org.apache.flume.sink.hdfs.HDFSEventSink.process(HDFSEventSink.java:380)
        at org.apache.flume.sink.DefaultSinkProcessor.process(DefaultSinkProcessor.java:68)
        at org.apache.flume.SinkRunner$PollingRunner.run(SinkRunner.java:147)
        at java.lang.Thread.run(Thread.java:745)

原因：

kafka生产者没有提供时间戳。

解决：

一、修改flume配置文件

添加：k2h.sinks.k1.hdfs.useLocalTimeStamp=true

二、kafka 生产者添加时间戳

new ProducerRecord<String, String>(topic,null,System.currentTimeMillis(), UUID.randomUUID().toString(), messageStr)