自定义Flink时间窗口
初学Flink,仅供参考,欢迎指正。
需求说明
Flink提供的常用简单窗口有:TumblingEventTimeWindows,TumblingProcessingTimeWindows,SlidingEventTimeWindows,SlidingProcessingTimeWindows等。对于初学者来说,这些窗口并不能满足特定的需求,例如:当接收到活动告警后,延迟一分钟,这一分钟内如果有对应清除告警,则当前告警不做通知。此时,我需要的是一个以活动告警的发生时间为起始时间,长度一分钟的窗口,而以上窗口无法满足。
- 触发器和驱逐器 触发器及驱逐器,本文不再赘述,采用默认的时间触发和驱逐;
- 窗口指派 通过重写WindowAssigner的assignWindows方法,可以定义需要的窗口;
自定义窗口
我测试使用的一个Demo, 代码如下
.
class TestEventTimeWindows extends WindowAssigner<Object, TimeWindow> {
private static final long serialVersionUID = 1L;
private final long size;
protected TestEventTimeWindows(long size) {
if (size > 0L) {
this.size = size;
} else {
throw new IllegalArgumentException("TestEventTimeWindows parameters must satisfy size > 0");
}
}
@Override
public Collection<TimeWindow> assignWindows(Object element, long timestamp, WindowAssignerContext context) {
if (timestamp <= -9223372036854775808L) {
throw new RuntimeException("Record has Long.MIN_VALUE timestamp (= no timestamp marker). Is the time characteristic set to 'ProcessingTime', or did you forget to call 'DataStream.assignTimestampsAndWatermarks(...)'?");
} else {
Tuple2<String, Map<String, Object>> alarm = (Tuple2) element;
List<TimeWindow> windows = new ArrayList(1);
// 判断告警状态,活动告警创建对应长度窗口
if ("1".equals(alarm.f1.get("status").toString())) {
long lastStart = Long.parseLong(alarm.f1.get("Time").toString());
windows.add(new TimeWindow(lastStart, lastStart + this.size));
}
return windows;
}
}
public long getSize() {
return this.size;
}
public Trigger<Object, TimeWindow> getDefaultTrigger(StreamExecutionEnvironment env) {
return EventTimeTrigger.create();
}
public String toString() {
return "TestEventTimeWindows(" + this.size + ")";
}
public static TestEventTimeWindows of(Time size) {
return new TestEventTimeWindows(size.toMilliseconds());
}
public TypeSerializer<TimeWindow> getWindowSerializer(ExecutionConfig executionConfig) {
return new TimeWindow.Serializer();
}
public boolean isEventTime() {
return true;
}
}
通过KeyedStream的window()指定已定义的窗口:
WindowedStream<Tuple2<String, Map<String, Object>>, Tuple, TimeWindow> timeWindow = tuple2TupleKeyedStream
.window(TestEventTimeWindows.of(Time.of(4, TimeUnit.MINUTES)));
以上内容仅供参考,如有不当之处还请指正,初次写文章,敬请谅解。