KafkaStream时间戳问题CreateTime = -1引起的程序中断

Exception in thread "app-8835188a-e0a0-46da-ac2a-6820ec197628-StreamThread-1" org.apache.kafka.streams.errors.StreamsException: Input record ConsumerRecord(topic = raw_103, partition = 1, offset = 7032668, CreateTime = -1, serialized key size = -1, serialized value size = 111, headers = RecordHeaders(headers = [], isReadOnly = false), key = null, value = { "key1": [ 103, "4113471085724846255", "--", "2018-04-17 21:33:53" ], "key2": [ [ 213309, "--", 20128, 1 ] ] }) has invalid (negative) timestamp. Possibly because a pre-0.10 producer client was used to write this record to Kafka without embedding a timestamp, or because the input topic was created before upgrading the Kafka cluster to 0.10+. Use a different TimestampExtractor to process this data.
        at org.apache.kafka.streams.processor.FailOnInvalidTimestamp.onInvalidTimestamp(FailOnInvalidTimestamp.java:73)
        at org.apache.kafka.streams.processor.ExtractRecordMetadataTimestamp.extract(ExtractRecordMetadataTimestamp.java:61)
        at org.apache.kafka.streams.processor.FailOnInvalidTimestamp.extract(FailOnInvalidTimestamp.java:48)
        at org.apache.kafka.streams.processor.internals.RecordQueue.addRawRecords(RecordQueue.java:98)
        at org.apache.kafka.streams.processor.internals.PartitionGroup.addRawRecords(PartitionGroup.java:117)
        at org.apache.kafka.streams.processor.internals.StreamTask.addRecords(StreamTask.java:560)
        at org.apache.kafka.streams.processor.internals.StreamThread.addRecordsToTasks(StreamThread.java:896)
        at org.apache.kafka.streams.processor.internals.StreamThread.runOnce(StreamThread.java:797)
        at org.apache.kafka.streams.processor.internals.StreamThread.runLoop(StreamThread.java:750)
        at org.apache.kafka.streams.processor.internals.StreamThread.run(StreamThread.java:720)

之前直接改了源码。后来从度娘中找到解决方法:
新增时间异常捕获类MyEventTimeExtractor.class, 直接返回0

import org.apache.kafka.clients.consumer.ConsumerRecord;
import org.apache.kafka.streams.processor.TimestampExtractor;

public class MyEventTimeExtractor implements TimestampExtractor{
    @Override
    public long extract(ConsumerRecord<Object, Object> record,
            long previousTimestamp) {
        return 0;
    }
}

然后在属性添加下面配置:

props.put(StreamsConfig.DEFAULT_TIMESTAMP_EXTRACTOR_CLASS_CONFIG, MyEventTimeExtractor.class);

编译执行,ok

  • 1
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
要将JavaScript的时间戳转换为"yy-mm-dd"格式,可以使用以下方法之一。 方法1: 使用Date对象的方法进行转换。首先,创建一个Date对象,并将时间戳作为参数传递给它。然后,使用Date对象的get方法获取年、月和日,并进行格式化拼接。 ```javascript let timestamp = 1627085595000; // 时间戳 let date = new Date(timestamp); let year = date.getFullYear().toString().slice(-2); // 获取年份的后两位 let month = (date.getMonth() + 1).toString().padStart(2, '0'); // 获取月份,并补齐两位 let day = date.getDate().toString().padStart(2, '0'); // 获取日期,并补齐两位 let formattedDate = `${year}-${month}-${day}`; console.log(formattedDate); // 输出: 21-07-24 ``` 方法2: 使用自定义函数将时间戳转换为日期字符串。在函数中,使用Date对象的方法获取年、月和日,并进行格式化拼接。 ```javascript function timestampToDateString(timestamp) { let date = new Date(timestamp); let year = date.getFullYear().toString().slice(-2); // 获取年份的后两位 let month = (date.getMonth() + 1).toString().padStart(2, '0'); // 获取月份,并补齐两位 let day = date.getDate().toString().padStart(2, '0'); // 获取日期,并补齐两位 return `${year}-${month}-${day}`; } let timestamp = 1627085595000; // 时间戳 let formattedDate = timestampToDateString(timestamp); console.log(formattedDate); // 输出: 21-07-24 ``` 这两种方法都可以将JavaScript的时间戳转换为"yy-mm-dd"格式的日期字符串。<span class="em">1</span><span class="em">2</span><span class="em">3</span><span class="em">4</span>
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值