flink中的join,coGroup和intervalJoin

1.join

join() 算子提供的语义为"Window join"，即按照指定字段和（滚动/滑动/会话）窗口进行 inner join，支持处理时间和事件时间两种时间特征,只要其中一个数据流中没有数据就不用打印出来

DataStream  infoDataStream = . . . 
DataStream  infoDataStream1 = . . . 
DataStream<String> apply = infoDataStream.join(infoDataStream1).where(info -> info.getOrder_no()).equalTo(in -> in.getRequestId())
                .window(TumblingProcessingTimeWindows.of(Time.seconds(10)))
                .apply(new JoinFunction<Info, UserInfo, String>() {
                    @Override
                    public String join(Info info, UserInfo userInfo) throws Exception {
                        return info.getOrder_no() + "==========" + userInfo.getRequestId();
                   }
                });
        apply.print();

2.coGroup

要实现 left/right outer join 可以利用 coGroup() 算子。它的调用方式类似于 join() 算子，也需要开窗，但是 CoGroupFunction 比 JoinFunction 更加灵活，可以按照用户指定的逻辑匹配左流和/或右流的数据并输出

DataStream<Tuple2<String, String>> apply = infoDataStream1.coGroup(infoDataStream).where(info -> info.getRequestId()).equalTo(in -> in.getOrder_no())
                .window(TumblingProcessingTimeWindows.of(Time.seconds(10)))
                .apply(new CoGroupFunction<UserInfo, Info, Tuple2<String, String>>() {
                    @Override
                    public void coGroup(Iterable<UserInfo> iterable, Iterable<Info> iterable1, Collector<Tuple2<String, String>> collector) throws Exception {
                        for (UserInfo in : iterable) {
                            boolean isMatched = false;
                            for (Info info : iterable1) {
                                // 右流中有对应的记录
                                collector.collect(new Tuple2<String, String>(in.getRecordCount(), info.getOrder_no()));
                                isMatched = true;
                            }
                            if (!isMatched) {
                                // 右流中没有对应的记录
                                collector.collect(new Tuple2<>(in.getRecordCount(), null));
                            }
                        }
                    }
                });
        apply.print();

3.intervalJoin

join() 和 coGroup() 都是基于窗口做关联的。但是在某些情况下，两条流的数据步调未必一致。例如，订单流的数据有可能在点击流的购买动作发生之后很久才被写入，如果用窗口来圈定，很容易 join 不上。所以 Flink 又提供了"Interval join"的语义，按照指定字段以及右流相对左流偏移的时间区间进行关联，即：

right.timestamp ∈ [left.timestamp + lowerBound; left.timestamp + upperBound]

在运行之前，需要分别在两个流上应用 assignTimestampsAndWatermarks() 方法获取事件时间戳和水印

SingleOutputStreamOperator<String> process = infoDataStream1.keyBy(info -> info.getRequestId()).intervalJoin(infoDataStream.keyBy(in -> in.getOrder_no()))
                // 时间间隔,设定下界和上界
                .between(Time.seconds(-30), Time.seconds(30))
                .process(new ProcessJoinFunction<UserInfo, Info, String>() {
                    @Override
                    public void processElement(UserInfo left, Info right, Context ctx, Collector<String> out) throws Exception {
                        out.collect(StringUtils.join(Arrays.asList(left.getRecordCount(),
                                right.getOrder_no(), right.getName()),
                                '\t'));
                    }
                });
        process.print();