Flink跟着问题读源码 - SlidingEventTimeWindows接reduce结果数据倍增

最新推荐文章于 2022-10-15 23:11:04 发布

axin1995

最新推荐文章于 2022-10-15 23:11:04 发布

阅读量1k

点赞数

分类专栏： Flink学习文章标签： flink

本文链接：https://blog.csdn.net/m0_49826240/article/details/108823781

版权

1.异常情况

使用 SlidingEventTimeWindows 窗口，后面直接一个reduce算子，数据会出现倍增情况。

代码：

DataStreamSource<String> localhost = env.socketTextStream("localhost", 9991, "\n");

final SingleOutputStreamOperator<DataEntity> dataSteam = localhost
		.flatMap(new FlatMapFunction<String, DataEntity>() {
			@Override
			public void flatMap(String value, Collector<DataEntity> out) throws Exception {

				try {
					DataEntity dataEntity = new DataEntity();
					dataEntity.setCount(1);
					dataEntity.setKey("121212");
					dataEntity.setEventTime(System.currentTimeMillis());
					out.collect(dataEntity);
				} catch (Exception e) {
					int p = 0;
				}
			}
		});

dataSteam.print("source-");

SingleOutputStreamOperator<DataEntity> datawmStream = dataSteam
		.assignTimestampsAndWatermarks(new BoundedOutOfOr