接口Function -》接口FilterFunction、MapFunction、FlatMapFunction、ReduceFunction、CoMapFunction
自定义Function,实现以上函数,并重写内部的方法。
ds.reduce(new myFunction());
class myFunction implements ReduceFunction<SensorReading>() {
@Override
public SensorReading reduce(SensorReading value1, SensorReading value2) throws Exception {
return new SensorReading(value1.getId(), value2.getTimestamp(), Math.max(value1.getTemperature(),value2.getTemperature()));
}
})
或者写成匿名类
DataStream result = keyedStream.reduce(new ReduceFunction<SensorReading>() {
@Override
public SensorReading reduce(SensorReading value1, SensorReading value2) throws Exception {
return new SensorReading(value1.getId(), value2.getTimestamp(), Math.max(value1.getTemperature(),value2.getTemperature()));
}
});
- lambda函数
DataStream result = keyedStream.reduce((v1, v2) -> v1+v2 );
4,Rich Function
RichMapFunction
RichFlapMapFunction
RichFilterFunction
…
与常规函数不同的是,可以获取运行环境上下文,并通过生命周期方法实现更复杂功能。
生命周期方法:
- open(), 算子如map执行前会被调用
- close(), 清理工作
- getRuntimeContext() ,运行时上下文,如并行度、
class MyFunction extends RichMapFunction<SensorReading, Tuple2<String, Integer>> {
@Override
public void setRuntimeContext(RuntimeContext t) {
super.setRuntimeContext(t);
}
@Override
public RuntimeContext getRuntimeContext() {
return super.getRuntimeContext();
}
@Override
public IterationRuntimeContext getIterationRuntimeContext() {
return super.getIterationRuntimeContext();
}
@Override
public void open(Configuration parameters) throws Exception {
super.open(parameters);
}
@Override
public void close() throws Exception {
super.close();
}
@Override
public Tuple2<String, Integer> map(SensorReading value) throws Exception {
value
return null;
}
}