flink 学习(九)flink会话窗口


前言

        会话窗口没有固定的开始和结束时间,数据不会重叠。在一段时间内没有接收到数据时当前会话窗口会关闭。

会话窗口包括:

ProcessingTimeSessionWindows.withGap
ProcessingTimeSessionWindows.withDynamicGap

EventTimeSessionWindows.withGap
EventTimeSessionWindows.withDynamicGap

DynamicEventTimeSessionWindows
DynamicProcessingTimeSessionWindows.withDynamicGap

静态与动态的区别:

静态可以指定gap的大小,

动态需要通过SessionWindowTimeGapExtractor接口获取动态的gap。

1.ProcessingTime-Session-Window-WithGap

基于数据处理时间的会话窗口,固定时间间隔
(1)数据源

public class IntegerSource implements SourceFunction<Integer> {
    int i = 0;

    @Override
    public void run(SourceContext ctx) throws Exception {
        while (true) {
            ctx.collect(i++);

            //调整数据生产时间间隔
            if (i % 4 == 0) {
                Thread.sleep(4000);
            } else {
                Thread.sleep(1000);
            }

        }
    }

    @Override
    public void cancel() {

    }
}

(2)示例

 	@Test
    public void processingTimeSessionWindowsWithGapTest() throws Exception {
        SimpleDateFormat format = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
        final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
        env.setRuntimeMode(RuntimeExecutionMode.STREAMING);
        DataStreamSource<Integer> source = env.addSource(new IntegerSource());
        //基于ProcessingTime的会话窗口,时间间隔是3s
        source.windowAll(ProcessingTimeSessionWindows.withGap(Time.seconds(3)))
                .process(new ProcessAllWindowFunction<Integer, Integer, TimeWindow>() {
                    @Override
                    public void process(Context context, Iterable<Integer> elements, Collector<Integer> out) throws Exception {
                        Iterator<Integer> it = elements.iterator();
                        int sum = 0;
                        while (it.hasNext()) {
                            Integer next = it.next();
                            sum += next;
                            System.out.println("元素: " + next + " ,处理时间:" + format.format(new Date()));
                        }
                        out.collect(sum);
                    }
                })
                .print();
        env.execute("processingTimeSessionWindowsWithGap");
    }

结果:

元素: 0 ,处理时间:2022-04-17 22:41:49
元素: 1 ,处理时间:2022-04-17 22:41:49
元素: 2 ,处理时间:2022-04-17 22:41:49
元素: 3 ,处理时间:2022-04-17 22:41:49
4> 6
元素: 4 ,处理时间:2022-04-17 22:41:56
元素: 5 ,处理时间:2022-04-17 22:41:56
元素: 6 ,处理时间:2022-04-17 22:41:56
元素: 7 ,处理时间:2022-04-17 22:41:56
5> 22

2.EventTime-Session-Window-WithGap

基于数据处理时间的会话窗口,动态时间间隔

(1)数据源

public class EventElementSource implements SourceFunction<EventElement> {
    @Override
    public void run(SourceContext ctx) throws Exception {
        int id = 0;
        long time = new Date().getTime();
        while (true) {
            EventElement eventElement1 = new EventElement(++id, time, new Date(time));
            time += 1999;
            EventElement eventElement2 = new EventElement(++id, time, new Date(time));
            time += 2000;
            EventElement eventElement3 = new EventElement(++id, time, new Date(time));
            time += 3000;
            EventElement eventElement4 = new EventElement(++id, time, new Date(time));
            time += 4000;
            EventElement eventElement5 = new EventElement(++id, time, new Date(time));
            ctx.collect(eventElement1);
            ctx.collect(eventElement2);
            ctx.collect(eventElement3);
            ctx.collect(eventElement4);
            ctx.collect(eventElement5);
            Thread.sleep(2000);
        }

    @Override
    public void cancel() {

    }
}

(2)示例

 public void eventTimeSessionWindowsTest() throws Exception {
        SimpleDateFormat format = new SimpleDateFormat("yyyy-MM-dd HH:mm:ss");
        final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();
        env.setRuntimeMode(RuntimeExecutionMode.STREAMING)
                .setParallelism(1);
//        env.getConfig().setAutoWatermarkInterval(1000);

        //添加数据源
        env.addSource(new EventElementSource())
                //设置时间戳和水印
                .assignTimestampsAndWatermarks(WatermarkStrategy.<EventElement>forMonotonousTimestamps().withTimestampAssigner((eventTElement, re) -> eventTElement.getTime()))
                //基于事件时间的会话窗口
                .windowAll(EventTimeSessionWindows.withGap(Time.seconds(2)))
                .process(new ProcessAllWindowFunction<EventElement, Integer, TimeWindow>() {
                    @Override
                    public void process(Context context, Iterable<EventElement> elements, Collector<Integer> out) throws Exception {
                        Iterator<EventElement> it = elements.iterator();
                        int id = 0;
                        while (it.hasNext()) {
                            EventElement next = it.next();
                            id = next.getId();
                            System.out.println("元素: " + next + " ,处理时间:" + format.format(new Date()));
                        }
                        out.collect(id);
                    }
                })
                .print();
        env.execute("EventTimeSessionWindows");
    }

结果:数据时间间隔大于等于3s会开始新的会话窗口

元素: EventElement(id=1, time=1650206708180, date=Sun Apr 17 22:45:08 CST 2022) ,处理时间:2022-04-17 22:45:08
元素: EventElement(id=2, time=1650206710179, date=Sun Apr 17 22:45:10 CST 2022) ,处理时间:2022-04-17 22:45:08
元素: EventElement(id=3, time=1650206712179, date=Sun Apr 17 22:45:12 CST 2022) ,处理时间:2022-04-17 22:45:08
3
元素: EventElement(id=4, time=1650206715179, date=Sun Apr 17 22:45:15 CST 2022) ,处理时间:2022-04-17 22:45:08
4
元素: EventElement(id=5, time=1650206719179, date=Sun Apr 17 22:45:19 CST 2022) ,处理时间:2022-04-17 22:45:10
元素: EventElement(id=6, time=1650206719179, date=Sun Apr 17 22:45:19 CST 2022) ,处理时间:2022-04-17 22:45:10
元素: EventElement(id=7, time=1650206721178, date=Sun Apr 17 22:45:21 CST 2022) ,处理时间:2022-04-17 22:45:10
元素: EventElement(id=8, time=1650206723178, date=Sun Apr 17 22:45:23 CST 2022) ,处理时间:2022-04-17 22:45:10
8
元素: EventElement(id=9, time=1650206726178, date=Sun Apr 17 22:45:26 CST 2022) ,处理时间:2022-04-17 22:45:10
9
元素: EventElement(id=10, time=1650206730178, date=Sun Apr 17 22:45:30 CST 2022) ,处理时间:2022-04-17 22:45:12
元素: EventElement(id=11, time=1650206730178, date=Sun Apr 17 22:45:30 CST 2022) ,处理时间:2022-04-17 22:45:12
元素: EventElement(id=12, time=1650206732177, date=Sun Apr 17 22:45:32 CST 2022) ,处理时间:2022-04-17 22:45:12
元素: EventElement(id=13, time=1650206734177, date=Sun Apr 17 22:45:34 CST 2022) ,处理时间:2022-04-17 22:45:12
13
  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 打赏
    打赏
  • 0
    评论
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

_lrs

你的鼓励将是我创作的最大动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付:¥1
获取中
扫码支付

您的余额不足,请更换扫码支付或充值

打赏作者

实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值