flink设置watermark以及事件时间字段源码分析

flink设置watermark以及事件时间字段源码分析

背景

1.1、提取时间戳字段，用于事件时间语义处理数据

1.2、设置水位线(水印)watermark

TimestampAssigner 核心接口介绍

TimestampAssigner 时间分配器接口 实现类关系图：

提取时间戳字段方法：

TimestampAssigner 时间戳分配器, 提取数据流中的时间戳字段，

public interface TimestampAssigner<T> extends Function {
   

	/**
	 * Assigns a timestamp to an element, in milliseconds since the Epoch.
	 *
	 * <p>The method is passed the previously assigned timestamp of the element.
	 * That previous timestamp may have been assigned from a previous assigner,
	 * by ingestion time. If the element did not carry a timestamp before, this value is
	 * {@code Long.MIN_VALUE}.
	 *
	 * @param element The element that the timestamp will be assigned to.
	 * @param previousElementTimestamp The previous internal timestamp of the element,
	 *                                 or a negative value, if no timestamp has been assigned yet.
	 * @return The new timestamp.
	 */
	long extractTimestamp(T element, long previousElementTimestamp);
}

TimestampAssigner 实现类

AssignerWithPeriodicWatermarks  //周期性的生成水印
AssignerWithPunctuatedWatermarks //打断式的生成，也就是可以每一条数据都生成
BoundedOutOfOrdernessTimestampExtractor //乱序数据周期性生成
AscendingTimestampExtractor //升序数据周期性生成
IngestionTimeExtractor  //进入flink系统时间分配器

设置时间戳、水印方法

DataStream类设置时间戳的方法：assignTimestampsAndWatermarks，指定watermark

	public SingleOutputStreamOperator<T> assignTimestampsAndWatermarks(
			AssignerWithPeriodicWatermarks<T> timestampAndWatermarkAssigner) {
   

		// match parallelism to input, otherwise dop=1 sources could lead to some strange
		// behaviour: the watermark will creep along very slowly because the elements
		// from the source go to each extraction operator round robin.
		final int inputParallelism = getTransformation().getParallelism();
		final AssignerWithPeriodicWatermarks<T> cleanedAssigner = clean(timestampAndWatermarkAssigner);

		TimestampsAndPeriodicWatermarksOperator<T> operator =
				new TimestampsAndPeriodicWatermarksOperator<>(cleanedAssigner);

		return transform("Timestamps/Watermarks", getTransformation().getOutputType(), operator)
				.setParallelism(inputParallelism);
	}

---

---

1、AssignerWithPeriodicWatermarks接口：

public interface AssignerWithPeriodicWatermarks<T