flink interval join源码分析

Interval Join

Returns a simple Cartesian product restricted by the join condition and a time constraint. An interval join requires at least one equi-join predicate and a join condition that bounds the time on both sides. Two appropriate range predicates can define such a condition (<, <=, >=, >), a BETWEEN predicate, or a single equality predicate that compares time attributes of the same type (i.e., processing time or event time) of both input tables.

For example, this query will join all orders with their corresponding shipments if the order was shipped four hours after the order was received.

SELECT *
FROM Orders o, Shipments s
WHERE o.id = s.order_id
AND o.order_time BETWEEN s.ship_time - INTERVAL '4' HOUR AND s.ship_time + INTERVAL '8' HOUR

• ltime = rtime
• ltime >= rtime AND ltime < rtime + INTERVAL '10' MINUTE
• ltime BETWEEN rtime - INTERVAL '10' SECOND AND rtime + INTERVAL '5' SECOND

watermark

TwoInputStreamOperator算子。

public void processWatermark1(Watermark mark) throws Exception {
    input1Watermark = mark.getTimestamp();
    long newMin = Math.min(input1Watermark, input2Watermark);
    if (newMin > combinedWatermark) {
        combinedWatermark = newMin;
        processWatermark(new Watermark(combinedWatermark));
    }
}

watermark取两条流中更小的那个watermark，input1Watermark, input2Watermark这两个watermark是各自流里面最新的watermark。这里有两个逻辑，一个是各自流中的watermark，一个是两条流watermark的合并，不要混淆了。

ProcessElement1

左流和右流是对应的关系。

1. 更新operatorTime

void updateOperatorTime(Context ctx) {
    leftOperatorTime =
        ctx.timerService().currentWatermark() > 0
        ? ctx.timerService().currentWatermark()
        : 0L;
    // We may set different operator times in the future.
    rightOperatorTime = leftOperatorTime;
}

2. 获取事件的eventTime
3. 确定右边流的上下界

leftRelativeSize = -leftLowerBound;
rightRelativeSize = leftUpperBound;

long rightQualifiedLowerBound = timeForLeftRow - rightRelativeSize;
long rightQualifiedUpperBound = timeForLeftRow + leftRelativeSize;

4. 确定右边流的过期时间，这里的过期指的是太早以前的数据

rightExpirationTime = leftOperatorTime - rightRelativeSize - allowedLateness - 1

5. 获取右边流的缓存

Iterator<Map.Entry<Long, List<Tuple2<RowData, Boolean>>>> rightIterator =
                    rightCache.iterator();

6. 迭代右边缓存流里面的数据

while (rightIterator.hasNext()) {
    Map.Entry<Long, List<Tuple2<RowData, Boolean>>> rightEntry = rightIterator.next();
    Long rightTime = rightEntry.getKey();
    if (rightTime >= rightQualifiedLowerBound
        && rightTime <= rightQualifiedUpperBound) {
        List<Tuple2<RowData, Boolean>> rightRows = rightEntry.getValue();
        boolean entryUpdated = false;
        for (Tuple2<RowData, Boolean> tuple : rightRows) {
            joinCollector.reset();
            joinFunction.join(leftRow, tuple.f0, joinCollector);
            emitted = emitted || joinCollector.isEmitted();
            if (joinType.isRightOuter()) {
                if (!tuple.f1 && joinCollector.isEmitted()) {
                    // Mark the right row as being successfully joined and emitted.
                    tuple.f1 = true;
                    entryUpdated = true;
                }
            }
        }
        if (entryUpdated) {
            // Write back the edited entry (mark emitted) for the right cache.
            rightEntry.setValue(rightRows);
        }
    }
    // Clean up the expired right cache row, clean the cache while join
    if (rightTime <= rightExpirationTime) {
        if (joinType.isRightOuter()) {
            List<Tuple2<RowData, Boolean>> rightRows = rightEntry.getValue();
            rightRows.forEach(
                (Tuple2<RowData, Boolean> tuple) -> {
                    if (!tuple.f1) {
                        // Emit a null padding result if the right row has never
                        // been successfully joined.
                        joinCollector.collect(paddingUtil.padRight(tuple.f0));
                    }
                });
        }
        // eager remove
        rightIterator.remove();
    } // We could do the short-cutting optimization here once we get a state with
    // ordered keys.
}
}

7. 缓存当前的数据，并且注册清除时间

if (rightOperatorTime < rightQualifiedUpperBound) {
    // Operator time of right stream has not exceeded the upper window bound of the current
    // row. Put it into the left cache, since later coming records from the right stream are
    // expected to be joined with it.
    List<Tuple2<RowData, Boolean>> leftRowList = leftCache.get(timeForLeftRow);
    if (leftRowList == null) {
        leftRowList = new ArrayList<>(1);
    }
    leftRowList.add(Tuple2.of(leftRow, emitted));
    leftCache.put(timeForLeftRow, leftRowList);
    if (rightTimerState.value() == null) {
        // Register a timer on the RIGHT stream to remove rows.
        registerCleanUpTimer(ctx, timeForLeftRow, true);
    }
} else if (!emitted && joinType.isLeftOuter()) {
    // Emit a null padding result if the left row is not cached and successfully joined.
    joinCollector.collect(paddingUtil.padLeft(leftRow));
}

8. 清除机制

private void registerCleanUpTimer(Context ctx, long rowTime, boolean leftRow)
    throws IOException {
    if (leftRow) {
        long cleanUpTime =
            rowTime + leftRelativeSize + minCleanUpInterval + allowedLateness + 1;
        registerTimer(ctx, cleanUpTime);
        rightTimerState.update(cleanUpTime);
    } else {
        long cleanUpTime =
            rowTime + rightRelativeSize + minCleanUpInterval + allowedLateness + 1;
        registerTimer(ctx, cleanUpTime);
        leftTimerState.update(cleanUpTime);
    }
}

minCleanUpInterval = (leftRelativeSize + rightRelativeSize) / 2;

详解

SELECT *
FROM Orders o, Shipments s
WHERE o.id = s.order_id
AND o.order_time BETWEEN s.ship_time - INTERVAL '5' HOUR AND s.ship_time + INTERVAL '10' HOUR

触发

触发很简单，就是左边来了一条数据，去找右边流的缓存，根据匹配结果输出数据。反过来就是右边来了一条数据，去找左边流的缓存。

缓存

根据上面的例子可以得到两个公式：

s.t - 5 <= o.t <= s.t + 10

o.t - 10 <= s.t <= o.t + 5

假设s表的水位线为18，o表进来了一条数据，事件时间为20，带入公式中，那么右表的有效范围为：[10, 25]

缓存的条件是当前的watermark < o.t + 5, 18 < 25所以o表的这条数据会存入o表的缓存。

这里的缓存条件限制的是迟到的数据，对于后面进来的正常数据，这个条件一定满足的。因为当水位线为当前事件时间的时候，s.t的右边界为o.t + 5，o.t一定小于o.t +5；当事件时间没有更新水位线的时候，那么watermark < o.t < o.t + 5。所以正常的数据一定会被缓存。

只有当事件为迟到事件时，才会出现watermark > o.t + 5。比如左边流来了一条事件时间为12的数据，18 > 12 + 5，因此这条数据不会被左边流缓存。但是如果左边来了一条事件时间为17的数据，小于水位线，18 < 17 + 5，这条数据也会被左边流缓存。

例子：

create table leftTable (
  `row_time` TIMESTAMP(3),
  `num` int,
  `id` string,
  watermark for row_time as row_time - interval '1' second
) WITH (
  'connector' = 'kafka',
  'topic' = 'demo',
  'properties.bootstrap.servers' = 'bigdata03:9092',
  'properties.group.id' = 'testGroup',
  'scan.startup.mode' = 'latest-offset',
  'format' = 'csv'
);

create table rightTable (
  `row_time` TIMESTAMP(3),
  `num` int,
  `id` string,
  watermark for row_time as row_time - interval '1' second
) WITH (
  'connector' = 'kafka',
  'topic' = 'demo1',
  'properties.bootstrap.servers' = 'bigdata03:9092',
  'properties.group.id' = 'testGroup',
  'scan.startup.mode' = 'latest-offset',
  'format' = 'csv'
);

select a.row_time, a.num, b.id
from leftTable a inner join rightTable b
on a.num = b.num
and a.row_time between b.row_time - interval '5' minute and b.row_time + interval '10' minute;

// 左流
2020-04-15 12:20:00,4,L20
// 右流
2020-04-15 12:18:00,4,R18

// 左流
2020-04-15 12:11:00,4,L11
2020-04-15 12:17:00,4,L17

2020-04-15 12:11:00,4,L11并没有被左流缓存，因为这条数据对应的右流上界为``2020-04-15 12:16:00`

这个时候的水位线是2020-04-15 12:18:00，水位线大于右流上界，所以不缓存。

// 右流
2020-04-15 12:15:00,4,R15

可以验证2020-04-15 12:11:00,4,L11没有在左流的缓存中。

缓存清除

清除存储是一个定时任务

清除的时间为

minCleanUpInterval = (leftRelativeSize + rightRelativeSize) / 2;
cleanUpTime = rowTime + leftRelativeSize + minCleanUpInterval + allowedLateness + 1

// 左流
2020-04-15 12:10:00,4,L10
// 右流
2020-04-15 12:11:00,4,R11

// 左流
2020-04-15 12:40:00,4,L40
// 右流
2020-04-15 12:12:00,4,R12

这个时候水位线是在2020-04-15 12:12:00所以2020-04-15 12:10:00,4,L10没有被清除。

// 右流
2020-04-15 12:45:00,4,R45
// 右流
2020-04-15 12:13:00,4,R13

右流2020-04-15 12:45:00数据的进来导致水位线到达2020-04-15 12:40:00，触发清除左流的过期缓存。因此2020-04-15 12:13:00,4,R13没有匹配到数据。

清除的范围为事件事件小于leftExpirationTime的事件：

leftExpirationTime = operatorTime - leftRelativeSize - allowedLateness - 1

在这个例子里面就是

leftExpirationTime = 2020-04-15 12:40:00 - 5分钟 - 0 - 1毫秒

也就是左流中2020-04-15 12:34:59.999之前的数据被清除掉。