假设我有两个数据流,我已经从Kafka接收到了数据流,经过一些处理,我得到了这两个数据流。
DataStream A contains values {id1_id2 , id3_id4, id99_id0, id15_id3,id11_id5....}
DataStream B contains values {id2, id3,id5...}
是否可以对数据流A进行一些处理,以便它将值输出到另一个数据流中
DataStream C ={id1, id3, id15, id11}
因此,B中的所有值都将与A相交。我尝试过processElement()和RichCoFlatMapFunction,但它不起作用。
public class MatchAggregator
extends RichCoFlatMapFunction> {
private ValueState<String> doubleState;
private ValueState<String> singleState;
@Override
public void open(Configuration config) {
doubleState = getRuntimeContext().getState(new ValueStateDescriptor<>("doubleEvents",String.class));
singleState = getRuntimeContext().getState(new ValueStateDescriptor<>("singleEvents",String.class));
}
@Override
public void flatMap1(String s, Collector<Tuple1<String>> collector) throws Exception {
String single = singleState.value();
//this is outputting null.
System.out.println(single);
//s is also null
if(single.contains(s)){
String replaceNumber = single.replace(s,"");
String replaceEmp = replaceNumber.replace("_","");
single.clear();
collector.collect(Tuple1.of(replaceEmp));
}else {
personContactState.update(s);
}
}
@Override
public void flatMap2(String s, Collector<Tuple1<String>> collector) throws Exception {
}
}
我使用两个数据流,比如:
DataStream> match = A.connect(B).flatMap(new MatchAggregator());
match.print();