Flink原理实战每日一篇05

最新推荐文章于 2024-07-03 19:59:20 发布

黄瓜炖啤酒鸭

最新推荐文章于 2024-07-03 19:59:20 发布

阅读量296

点赞数

分类专栏： Flink 文章标签： Flink Trigger Evictors

本文链接：https://blog.csdn.net/qq_31866793/article/details/93314499

版权

本文深入探讨了Flink的窗口触发器（如EventTimeTrigger、ProcessTimeTrigger等）和数据剔除器（如CountEvictor、DeltaEvictor等）的原理及使用场景。介绍了如何自定义Trigger和Evictor，并讨论了延迟数据处理、连续窗口计算的概念，以及Windows的多流合并应用。同时，提到了Flink中不同类型的窗口关联方法，如滚动窗口、滑动窗口、会话窗口和间隔关联。

摘要由CSDN通过智能技术生成

接着上面一篇讲吧，下面要讲的东西实际上我基本没用过............好吧，只能硬着头皮再搞一遍，希望这次能彻底的弄明白trigger窗口触发器，以及 evictors 数据剔除器的原理跟使用场景~~~

1，Trigger 窗口触发器

拥有的触发器如下，自行百度~

EventTimeTrigger :

ProcessTimeTregger:

ContinuousEventTimeTregger:

CountTrigger:

DeltaTrigger:

PurgingTrigger:

如果以上发触发器都满足不了需求，可以自定义触发器：

继承并实现抽象类Trigger 自定义触发器，FlinkTrigger接口共有如下方法需要复写：

OnElement（）

OnEventTime（）

OnProcessTime ()

OnMerge ()

Clear ()

代码如下：

object TreiggerDemo {
  def main(args: Array[String]): Unit = {
    val env = StreamExecutionEnvironment.getExecutionEnvironment
    val dataStream: DataStream[(Int, Int)] = env.fromCollection(List((1, 2), (3, 4), (1, 3)))
    dataStream.keyBy(_._1)
//      .window(SlidingEventTimeWindows.of(Time.seconds(10), Time.seconds(2)))
      .timeWindow(Time.seconds(10))
        .trigger(ContinuousEventTimeTrigger.of(Time.seconds(5)))
    
    env.execute()
  }
}

调用的 ContinuousEventTimeTrigger类是自带的类：

public class ContinuousEventTimeTrigger<W extends Window> extends Trigger<Object, W> {
   private static final long serialVersionUID = 1L;

   private final long interval;

   /** When merging we take the lowest of all fire timestamps as the new fire timestamp. */
   private final ReducingStateDescriptor<Long> stateDesc =
         new ReducingStateDescriptor<>("fire-time", new Min(), LongSerializer.INSTANCE);

   private ContinuousEventTimeTrigger(long interval) {
      this.interval = interval;
   }

   @Override
   public TriggerResult onElement(Object element, long timestamp, W window, TriggerContext ctx) throws Exception {

      if (window.maxTimestamp() <= ctx.getCurrentWatermark()) {
         // if the watermark is already past the window fire immediately
         return TriggerResult.FIRE;
      } else {
         ctx.registerEventTimeTimer(window.maxTimestamp());
      }

      ReducingState<Long> fireTimestamp = ctx.getPartitionedState(stateDesc);
      if (fireTimestamp.get() == null) {
         long start = timestamp - (timestamp % interval);
         long nextFireTimestamp = start + interval;
         ctx.registerEventTimeTimer(nextFireTimestamp);
         fireTimestamp.add(nextFireTimestamp);
      }

      return TriggerResult.CONTINUE;
   }

   @Override
   public TriggerResult onEventTime(long time, W windo