flink 窗口函数简单案例

最新推荐文章于 2024-09-12 22:10:46 发布
最新推荐文章于 2024-09-12 22:10:46 发布
阅读量275 收藏
点赞数
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/jokertiger/article/details/118378663
版权

文章目录

SensorReading

package kgc.kb11.beans;

import java.util.ArrayList;

/**
 * @author zhouhu
 * @Date
 * @Desription
 */

public class SensorReading {
    private String id;
    private Long timestamp;
    private Double temperature;

    @Override
    public String toString() {
        return "SensorReading{" +
                "id='" + id + '\'' +
                ", timestamp=" + timestamp +
                ", temperature=" + temperature +
                '}';
    }

    public SensorReading(String id, Long timestamp, Double temperature) {
        this.id = id;
        this.timestamp = timestamp;
        this.temperature = temperature;
    }

    public SensorReading() {
    }

    public String getId() {
        return id;
    }

    public void setId(String id) {
        this.id = id;
    }

    public Long getTimestamp() {
        return timestamp;
    }

    public void setTimestamp(Long timestamp) {
        this.timestamp = timestamp;
    }

    public Double getTemperature() {
        return temperature;
    }

    public void setTemperature(Double temperature) {
        this.temperature = temperature;
    }

}

window1

package kgc.kb11.window;

import kgc.kb11.beans.SensorReading;
import org.apache.flink.api.common.functions.MapFunction;
import org.apache.flink.api.common.functions.ReduceFunction;
import org.apache.flink.api.common.serialization.SimpleStringSchema;
import org.apache.flink.streaming.api.datastream.DataStreamSource;
import org.apache.flink.streaming.api.datastream.KeyedStream;
import org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator;
import org.apache.flink.streaming.api.datastream.WindowedStream;
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
import org.apache.flink.streaming.api.windowing.assigners.TumblingEventTimeWindows;
import org.apache.flink.streaming.api.windowing.assigners.TumblingProcessingTimeWindows;
import org.apache.flink.streaming.api.windowing.time.Time;
import org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer011;
import org.apache.kafka.clients.consumer.ConsumerConfig;

import java.util.Properties;

/**
 * @author zhouhu
 * @Date
 * @Desription
 */

public class Window1 {
    public static void main(String[] args) throws Exception {
        StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();


        Properties prop = new Properties();
        prop.setProperty(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, "192.168.119.125:9092");
        prop.setProperty(ConsumerConfig.GROUP_ID_CONFIG, "sensor_group1");
        prop.setProperty(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, "org.apache.kafka.common.serialization.StringDeserializer");
        prop.setProperty(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, "org.apache.kafka.common.serialization.StringDeserializer");

        DataStreamSource<String> inputStream = env.addSource(new FlinkKafkaConsumer011<String>(
                "sensor",
                new SimpleStringSchema(),
                prop
        ));


        SingleOutputStreamOperator<SensorReading> mapStream = inputStream.map(new MapFunction<String, SensorReading>() {
            @Override
            public SensorReading map(String s) throws Exception {
                String[] splits = s.split(",");
                return new SensorReading(
                        splits[0],
                        Long.parseLong(splits[1]),
                        Double.parseDouble(splits[2])
                );
            }
        });
        SingleOutputStreamOperator<SensorReading> max = mapStream.keyBy("id")
//                .timeWindow(Time.seconds(15));//滚动
//        .timeWindow(Time.seconds(15),Time.seconds(5));//滑动窗口
//        .countWindow(6);
//        .countWindow(6,2);
//        .window(TumblingEventTimeWindows.of(Time.seconds(5))); //eventtime   事件发生时间
//        .window(TumblingProcessingTimeWindows.of(Time.seconds(5)));//processingtime 时间处理时间
//                .timeWindow(Time.seconds(15))
//                .max("temperature");
                .countWindow(6, 2)//6个元素一个 窗口，每两个计算一次
                .reduce(new ReduceFunction<SensorReading>() {
                    @Override
                    public SensorReading reduce(SensorReading value1, SensorReading value2) throws Exception {
                        return new SensorReading(value1.getId(),
                                value1.getTimestamp(),
                                value1.getTemperature()+value2.getTemperature()
                                );
                    }
                });


        max.print("max");


        env.execute("finkwindow");
    }
}

window2

package kgc.kb11.window;

import kgc.kb11.beans.SensorReading;
import org.apache.flink.api.common.functions.AggregateFunction;
import org.apache.flink.api.common.functions.MapFunction;
import org.apache.flink.api.common.functions.ReduceFunction;
import org.apache.flink.api.common.serialization.SimpleStringSchema;
import org.apache.flink.api.java.tuple.Tuple2;
import org.apache.flink.streaming.api.TimeCharacteristic;
import org.apache.flink.streaming.api.datastream.DataStreamSource;
import org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator;
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
import org.apache.flink.streaming.api.functions.timestamps.BoundedOutOfOrdernessTimestampExtractor;
import org.apache.flink.streaming.api.windowing.time.Time;
import org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer011;
import org.apache.kafka.clients.consumer.ConsumerConfig;

import java.util.Properties;

/**
 * @author zhouhu
 * @Date
 * @Desription
 */

public class Window2 {
    public static void main(String[] args) throws Exception {
        StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();


        Properties prop = new Properties();
        prop.setProperty(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, "192.168.119.125:9092");
        prop.setProperty(ConsumerConfig.GROUP_ID_CONFIG, "sensor_group1");
        prop.setProperty(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, "org.apache.kafka.common.serialization.StringDeserializer");
        prop.setProperty(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, "org.apache.kafka.common.serialization.StringDeserializer");

        DataStreamSource<String> inputStream = env.addSource(new FlinkKafkaConsumer011<String>(
                "sensor",
                new SimpleStringSchema(),
                prop
        ));


        SingleOutputStreamOperator<SensorReading> mapStream = inputStream.map(new MapFunction<String, SensorReading>() {
            @Override
            public SensorReading map(String s) throws Exception {
                String[] splits = s.split(",");
                return new SensorReading(
                        splits[0],
                        Long.parseLong(splits[1]),
                        Double.parseDouble(splits[2])
                );
            }
        });


        SingleOutputStreamOperator<Double> aggregate = mapStream.keyBy("id")
                .countWindow(6, 2)
                .aggregate(new MyAvgFunction());

        aggregate.print("max");


        env.execute("finkwindow");
    }

    private static class MyAvgFunction implements AggregateFunction<SensorReading,
            Tuple2<Double,Integer>,Double> {
        @Override
        public Tuple2<Double, Integer> createAccumulator() {
            return new Tuple2<>(0.0,0);
        }

        @Override
        public Tuple2<Double, Integer> add(SensorReading value, Tuple2<Double, Integer> accumulator) {
            Double temp = value.getTemperature() + accumulator.f0;
            Integer count = 1 + accumulator.f1;
            return new Tuple2<>(temp,count);
        }

        @Override
        public Double getResult(Tuple2<Double, Integer> accumulator) {
            return accumulator.f0/accumulator.f1;
        }

        @Override
        public Tuple2<Double, Integer> merge(Tuple2<Double, Integer> a, Tuple2<Double, Integer> b) {
            Double sumtemp =a.f0+b.f0;
            Integer sumcount=a.f1+b.f1;
            return new Tuple2<>(sumtemp,sumcount);
        }
    }
}

apply

package kgc.kb11.window;

import kgc.kb11.beans.SensorReading;
import org.apache.commons.collections.IteratorUtils;
import org.apache.flink.api.common.functions.MapFunction;
import org.apache.flink.api.common.serialization.SimpleStringSchema;
import org.apache.flink.api.java.tuple.Tuple;
import org.apache.flink.api.java.tuple.Tuple4;
import org.apache.flink.streaming.api.TimeCharacteristic;
import org.apache.flink.streaming.api.datastream.DataStreamSource;
import org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator;
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
import org.apache.flink.streaming.api.functions.timestamps.BoundedOutOfOrdernessTimestampExtractor;
import org.apache.flink.streaming.api.functions.windowing.WindowFunction;
import org.apache.flink.streaming.api.windowing.time.Time;
import org.apache.flink.streaming.api.windowing.windows.TimeWindow;
import org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer011;
import org.apache.flink.util.Collector;
import org.apache.kafka.clients.consumer.ConsumerConfig;

import java.util.Iterator;
import java.util.Properties;

/**
 * @author zhouhu
 * @Date
 * @Desription
 */

public class Window4 {
    public static void main(String[] args) throws Exception {
        StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();

        env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime);

        Properties prop = new Properties();
        prop.setProperty(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, "192.168.119.125:9092");
        prop.setProperty(ConsumerConfig.GROUP_ID_CONFIG, "sensor_group2");
        prop.setProperty(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, "org.apache.kafka.common.serialization.StringDeserializer");
        prop.setProperty(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, "org.apache.kafka.common.serialization.StringDeserializer");
        prop.setProperty(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "latest");

        DataStreamSource<String> inputStream = env.addSource(new FlinkKafkaConsumer011<String>(
                "sensor",
                new SimpleStringSchema(),
                prop
        ));


        SingleOutputStreamOperator<SensorReading> mapStream = inputStream.map(new MapFunction<String, SensorReading>() {
            @Override
            public SensorReading map(String s) throws Exception {
                String[] splits = s.split(",");
                return new SensorReading(
                        splits[0],
                        Long.parseLong(splits[1]),
                        Double.parseDouble(splits[2])
                );
            }
        })
                .assignTimestampsAndWatermarks(new BoundedOutOfOrdernessTimestampExtractor<SensorReading>(Time.seconds(0)) {
                    @Override
                    public long extractTimestamp(SensorReading element) {
                        return element.getTimestamp() * 1000L;
                    }
                });


        SingleOutputStreamOperator<Tuple4<String, Long, Long, Integer>> apply = mapStream.keyBy("id")
                .timeWindow(Time.seconds(5)) //,Time.seconds(5)
                .apply(new WindowFunction<SensorReading, Tuple4<String, Long, Long, Integer>, Tuple, TimeWindow>() {
                    @Override
                    public void apply(Tuple tuple, TimeWindow window, Iterable<SensorReading> input, Collector<Tuple4<String, Long, Long, Integer>> out) throws Exception {
                        String key = tuple.getField(0);
                        long start = window.getStart();
                        long end = window.getEnd();
                        Iterator<SensorReading> iterator = input.iterator();
                        int size = IteratorUtils.toList(iterator).size();
                        Tuple4<String, Long, Long, Integer> returnValue = new Tuple4<>(key, start, end, size);
                        out.collect(returnValue);
                    }
                });
        apply.print();


        env.execute("finkwindow1");
    }

}

window3

package kgc.kb11.window;

import kgc.kb11.beans.SensorReading;
import org.apache.flink.api.common.functions.AggregateFunction;
import org.apache.flink.api.common.functions.MapFunction;
import org.apache.flink.api.common.serialization.SimpleStringSchema;
import org.apache.flink.api.java.tuple.Tuple;
import org.apache.flink.api.java.tuple.Tuple2;
import org.apache.flink.streaming.api.TimeCharacteristic;
import org.apache.flink.streaming.api.datastream.DataStreamSource;
import org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator;
import org.apache.flink.streaming.api.datastream.WindowedStream;
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
import org.apache.flink.streaming.api.functions.timestamps.BoundedOutOfOrdernessTimestampExtractor;
import org.apache.flink.streaming.api.windowing.assigners.TumblingEventTimeWindows;
import org.apache.flink.streaming.api.windowing.time.Time;
import org.apache.flink.streaming.api.windowing.windows.TimeWindow;
import org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer011;
import org.apache.flink.util.OutputTag;
import org.apache.kafka.clients.consumer.ConsumerConfig;

import java.util.ArrayList;
import java.util.Properties;

import static jdk.nashorn.internal.objects.NativeMath.max;

/**
 * @author zhouhu
 * @Date
 * @Desription
 */

public class Window3 {
    public static void main(String[] args) throws Exception {
        StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();

        env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime);

        Properties prop = new Properties();
        prop.setProperty(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, "192.168.119.125:9092");
        prop.setProperty(ConsumerConfig.GROUP_ID_CONFIG, "sensor_group2");
        prop.setProperty(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, "org.apache.kafka.common.serialization.StringDeserializer");
        prop.setProperty(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, "org.apache.kafka.common.serialization.StringDeserializer");
        prop.setProperty(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "latest");

        DataStreamSource<String> inputStream = env.socketTextStream("192.168.119.125", 7777);
//        DataStreamSource<String> inputStream = env.addSource(new FlinkKafkaConsumer011<String>(
//                "sensor",
//                new SimpleStringSchema(),
//                prop
//        ));


        SingleOutputStreamOperator<SensorReading> mapStream = inputStream.map(new MapFunction<String, SensorReading>() {
            @Override
            public SensorReading map(String s) throws Exception {
                String[] splits = s.split(",");
                return new SensorReading(
                        splits[0],
                        Long.parseLong(splits[1]),
                        Double.parseDouble(splits[2])
                );
            }
        })
                .assignTimestampsAndWatermarks(new BoundedOutOfOrdernessTimestampExtractor<SensorReading>(Time.seconds(0)) {
                    @Override
                    public long extractTimestamp(SensorReading element) {
                        return element.getTimestamp() * 1000L;
                    }
                });


        OutputTag<SensorReading> outputTag = new OutputTag<>("late");

        SingleOutputStreamOperator<SensorReading> max = mapStream.keyBy("id")
//                .timeWindow(Time.seconds(3)) //,Time.seconds(5)

                .window(TumblingEventTimeWindows.of(Time.seconds(2)))

                .allowedLateness(Time.seconds(30))
                .sideOutputLateData(outputTag)
                .max("temperature");



        env.execute("finkwindow1");
    }

}

window4

package kgc.kb11.window;

import kgc.kb11.beans.SensorReading;
import org.apache.commons.collections.IteratorUtils;
import org.apache.flink.api.common.functions.MapFunction;
import org.apache.flink.api.common.serialization.SimpleStringSchema;
import org.apache.flink.api.java.tuple.Tuple;
import org.apache.flink.api.java.tuple.Tuple4;
import org.apache.flink.streaming.api.TimeCharacteristic;
import org.apache.flink.streaming.api.datastream.DataStreamSource;
import org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator;
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;
import org.apache.flink.streaming.api.functions.timestamps.BoundedOutOfOrdernessTimestampExtractor;
import org.apache.flink.streaming.api.functions.windowing.WindowFunction;
import org.apache.flink.streaming.api.windowing.time.Time;
import org.apache.flink.streaming.api.windowing.windows.TimeWindow;
import org.apache.flink.streaming.connectors.kafka.FlinkKafkaConsumer011;
import org.apache.flink.util.Collector;
import org.apache.kafka.clients.consumer.ConsumerConfig;

import java.util.Iterator;
import java.util.Properties;

/**
 * @author zhouhu
 * @Date
 * @Desription
 */

public class Window4 {
    public static void main(String[] args) throws Exception {
        StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();

        env.setStreamTimeCharacteristic(TimeCharacteristic.EventTime);

        Properties prop = new Properties();
        prop.setProperty(ConsumerConfig.BOOTSTRAP_SERVERS_CONFIG, "192.168.119.125:9092");
        prop.setProperty(ConsumerConfig.GROUP_ID_CONFIG, "sensor_group2");
        prop.setProperty(ConsumerConfig.KEY_DESERIALIZER_CLASS_CONFIG, "org.apache.kafka.common.serialization.StringDeserializer");
        prop.setProperty(ConsumerConfig.VALUE_DESERIALIZER_CLASS_CONFIG, "org.apache.kafka.common.serialization.StringDeserializer");
        prop.setProperty(ConsumerConfig.AUTO_OFFSET_RESET_CONFIG, "latest");

        DataStreamSource<String> inputStream = env.addSource(new FlinkKafkaConsumer011<String>(
                "sensor",
                new SimpleStringSchema(),
                prop
        ));


        SingleOutputStreamOperator<SensorReading> mapStream = inputStream.map(new MapFunction<String, SensorReading>() {
            @Override
            public SensorReading map(String s) throws Exception {
                String[] splits = s.split(",");
                return new SensorReading(
                        splits[0],
                        Long.parseLong(splits[1]),
                        Double.parseDouble(splits[2])
                );
            }
        })
                .assignTimestampsAndWatermarks(new BoundedOutOfOrdernessTimestampExtractor<SensorReading>(Time.seconds(0)) {
                    @Override
                    public long extractTimestamp(SensorReading element) {
                        return element.getTimestamp() * 1000L;
                    }
                });


        SingleOutputStreamOperator<Tuple4<String, Long, Long, Integer>> apply = mapStream.keyBy("id")
                .timeWindow(Time.seconds(5)) //,Time.seconds(5)
                .apply(new WindowFunction<SensorReading, Tuple4<String, Long, Long, Integer>, Tuple, TimeWindow>() {
                    @Override
                    public void apply(Tuple tuple, TimeWindow window, Iterable<SensorReading> input, Collector<Tuple4<String, Long, Long, Integer>> out) throws Exception {
                        String key = tuple.getField(0);
                        long start = window.getStart();
                        long end = window.getEnd();
                        Iterator<SensorReading> iterator = input.iterator();
                        int size = IteratorUtils.toList(iterator).size();
                        Tuple4<String, Long, Long, Integer> returnValue = new Tuple4<>(key, start, end, size);
                        out.collect(returnValue);
                    }
                });
        apply.print();


        env.execute("finkwindow1");
    }

}
jokertiger
关注
flink 常用窗口函数
congge_study的博客
01-16 2905
flink 常用窗口函数
flink--Window TVF【窗口化表值函数】讲解及源码示例
penriver的博客
11-25 2898
1. 什么是Window TVF Windowing TVFs： Flink定义的多态表函数(简称PTF)，用于将表中的元素划分到定义的窗口中 窗口化TVFs替代了 legacy Grouped Window Functions。 windows TVFs更符合SQL标准，更强大的支持复杂的窗口计算，例如Window TopN, Window Join【在1.13.x中还不支持】。 Grouped Window Functions【分组窗口函数】只能支持窗口聚合。 PTF 【Polymorphi
最全面的flink入门编程案例
09-18
本文的非常详细的介绍了 flink、如何进行大数据开发的，包含flink读取kafka、文本读取，hdfs
Flink 窗口函数（Window Functions）案例（结合增量聚合函数与全窗口函数场景）
weixin_45417821的博客
04-17 1090
网站里面，特别是电商场景，一般情况都要统计哪些是热门商品，或者哪个url的访问最大。 前提是根据每个url对应的访问量统计出来，然后再去排序，做其他选择 我们可以将页面访问量按照窗口，基于窗口划分，每个url它们分别被点击访问了几次 这里我们为了方便处理，单独定义了一个 POJO 类 UrlViewCount 来表示聚合输出结果的数据类型，包含了 url、浏览量以及窗口的起始结束时间。 Gitee中 public class UrlViewCount { public String url; //.
Flink窗口详解和各示例使用
长臂人猿的博客
08-05 2132
前言 Flink版本：1.12.1 将上面的数据类比于一个车流（带有一个值），如果你想获得所有车值的总和，那该怎么办呢？求和：不断的将数据相加，像聚合函数一样：Flink的常见算子和实例代码。  但是问题来了：实时数据流是不断的产生数据的，那么作为无界数据流，你永远不可能获得流的完整结果。也许你可以创建一个同样的求和数据流（无界）像这样： 关于事件时间和水印与窗口的联合使用及其demo代码：Flink事件时间和水印详解 小结 Flink 内置了几种类型的窗口分配器，如下图所示： 从大的类型来区分有:
云星数据---Apache Flink实战系列(精品版)】：Flink其他操作及内容002-Flink基于流的windowAndFunction操作001
云星数据-http://www.cloud-star.com.cn
11-22 806
一、流处理的基本概念1.全局窗口（Global Windows）1.1全局窗口概念1.用户上京东购物，会进行一系列的操作，比如（点击、浏览、搜索、购买、付款等），用户的操作可以被记录为用户操作数据流。 2.京东上的用户会同时有多个，每个用户的操作都是独立的，随机的，因此用户之间的行为没有必然联系，没有统一规律。 3.该窗口仅在有一个自定义trigger时才有用。否则由于global window没有
Flink 窗口函数（Window Functions）增量聚合函数案例 统计 PV/UV 这个比值
weixin_45417821的博客
04-17 1160
在电商网站中，PV（页面浏览量）和 UV（独立访客数）是非常重要的两个流量指标。一般来说，PV 统计的是所有的点击量；而对用户 id 进行去重之后，得到的就是 UV。所以有时我们会用 PV/UV 这个比值，来表示“人均重复访问量”，也就是平均每个用户会访问多少次页面，这在一定程度上代表了用户的粘度。 代码实现如下：Gitee中完整代码 public class WindowAggregateTest_PVUV { public static void main(String[] args) thr.
Flink windowAll 函数案例代码实操
qq_31866793的博客
08-28 4673
1，先创建一个生产者 模拟数据 package com.coder.flink.core.a_bilibili; import com.coder.flink.core.streaming2.custormSource.MyNoParalleSource; import org.apache.flink.api.common.serialization.SimpleStringSchema...
208.Flink（三）：窗口的使用，处理函数的使用
鹏哥哥Aaa
09-27 1567
窗口的使用，处理函数的使用
大数据Flink（一百一十七）：Flink SQL的窗口操作
Lansonli（蓝深李）的博客
09-12 1469
在流处理应用中，数据是连续不断的，因此我们不可能等到所有数据都到了才开始处理。当然我们可以每来一个消息就处理一次，但是有时我们需要做一些聚合类的处理，例如：在过去的1分钟内有多少用户点击了我们的网页。在这种情况下，我们必须定义一个窗口，用来收集最近一分钟内的数据，并对这个窗口内的数据进行计算。Flink 认为 Batch 是 Streaming 的一个特例，所以 Flink 底层引擎是一个流式引擎，在上面实现了流处理和批处理。而窗口（window）就是从 Streaming 到 Batch 的一个桥梁。
【Flink 实战篇】窗口实战：TUMBLE、HOP、SESSION
书山有路，学海无涯。记录成长，追逐梦想
03-23 2001
在流式计算中，流通常是无穷无尽的，我们无法知道什么时候数据源会继续 / 停止发送数据，所以在流上处理聚合事件（count、sum 等）的处理方式与批处理中的处理方式会有所差异。在流上一般用窗口（Window）来限定聚合的范围，例如 “过去 2 分钟网站点击量的计数”、“在最近 100 个人中点赞这个视频的总人数”。窗口的概念相当于帮我们收集了一张有限数据的动态表，我们可以对表中的数据进行聚合计算。
Flink的窗口计算案例
ZH519080的博客
04-15 3258
Flink的关于窗口算子的一般应用
Flink 处理函数应用案例
weixin_45417821的博客
04-18 495
文章目录应用案例——Top N使用 ProcessAllWindowFunction使用 KeyedProcessFunction 应用案例——Top N 窗口的计算处理，在实际应用中非常常见。对于一些比较复杂的需求，如果增量聚合函数无法满足，我们就需要考虑使用窗口处理函数这样的“大招”了。 网站中一个非常经典的例子，就是实时统计一段时间内的热门 url。例如，需要统计最近10 秒钟内最热门的两个 url 链接，并且每 5 秒钟更新一次。我们知道，这可以用一个滑动窗口来实现，而“热门度”一般可以直接用访问量
37、Flink 的窗口函数（Window Functions）详解
猫猫爱吃小鱼粮
05-21 1638
Flink 的窗口函数（Window Functions）详解
Flink中的窗口函数
weixin_40659514的博客
11-10 1794
flink中的窗口函数
Flink代码之窗口函数，求最大最小，平均值(三)
YellowXiuHui的博客
06-18 5283
一、窗口函数，分流后求温度最小值 一般需求的逻辑：分流，开窗，聚合 import com.atguigu.day2.SensorSource import org.apache.flink.streaming.api.scala._ import org.apache.flink.streaming.api.windowing.time.Time object MinTempPerWindow { def main(args: Array[String]): Unit = { val env
《初级》Flink 窗口函数练习
huzechen的博客
08-29 430
加微信拉微信交流群 微信：weixin605405145 ProcessWinFunOnWindow import org.apache.flink.api.java.tuple.Tuple; import org.apache.flink.api.java.tuple.Tuple3; import org.apache.flink.streaming.api.datastream.Dat...
flink中窗口函数(一)基础函数
qq_44665283的博客
03-29 1306
flink中窗口函数(一)基础函数 (1)ReduceFunction A ReduceFunction specifies how two elements from the input are combined to produce an output element of the same type. Flink uses a ReduceFunction to incrementally aggregate(增量聚合) the elements of a window. ReduceFunction
Pyflink教程(六)：窗口函数
yuxj的博客
03-14 1518
pyflink 窗口函数
flink sql 事件时间 窗口计算案例
最新发布
10-17
Flink SQL 的事件时间窗口计算是一种处理实时流数据的强大工具，它允许你基于事件发生的时间而不是传统的定时时间划分数据。例如，你可以创建滑动窗口，比如每5分钟滚动一次，或者Tumbling窗口，即固定长度的、不...
评论
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值