flink自定义数据源小小案例-Java版

案例

每秒种输出10个传感器上面的温度值。

代码

import com.zjc4j.bean.SensorReading;
import org.apache.flink.streaming.api.datastream.DataStream;
import org.apache.flink.streaming.api.datastream.DataStreamSource;
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;

import java.util.Arrays;

public class SourceTest2_MySource {
    public static void main(String[] args) throws Exception {
        StreamExecutionEnvironment executionEnvironment = StreamExecutionEnvironment.getExecutionEnvironment();
//        executionEnvironment.setParallelism(1);
        DataStream<SensorReading> objectDataStream = executionEnvironment.addSource(new MySensorSource());
        objectDataStream.print("输出").setParallelism(1);
        executionEnvironment.execute();
    }
}

import com.zjc4j.bean.SensorReading;
import org.apache.flink.streaming.api.functions.source.SourceFunction;

import java.util.HashMap;
import java.util.Random;

public class MySensorSource implements SourceFunction<SensorReading> {
    private boolean flag = true;
    @Override
    public void run(SourceContext<SensorReading> sourceContext) throws Exception {
        Random random = new Random();
        HashMap<String,Double> hashMap = new HashMap<>();
        for (int i = 0; i < 10; i++) {
            hashMap.put("sensor"+(i + 1),60 + random.nextGaussian() * 20);
        }

        while (flag) {
            System.out.println("--------------------------------------------------------------");
            for (String key : hashMap.keySet()) {
                sourceContext.collect(new SensorReading(key,System.currentTimeMillis(),hashMap.get(key) + random.nextGaussian()));
            }
            Thread.sleep(1000L);
        }
    }

    @Override
    public void cancel() {
        flag = false;

    }
}

结果：

--------------------------------------------------------------
输出> SensorReading{id='sensor4', timestamp=1637997843870, temperature=19.80964484236727}
输出> SensorReading{id='sensor5', timestamp=1637997843870, temperature=62.324935915709105}
输出> SensorReading{id='sensor2', timestamp=1637997843870, temperature=52.52693426840211}
输出> SensorReading{id='sensor3', timestamp=1637997843870, temperature=32.13924274749817}
输出> SensorReading{id='sensor10', timestamp=1637997843870, temperature=103.49680582215447}
输出> SensorReading{id='sensor8', timestamp=1637997843870, temperature=88.8881421381634}
输出> SensorReading{id='sensor9', timestamp=1637997843870, temperature=24.995045933529894}
输出> SensorReading{id='sensor6', timestamp=1637997843870, temperature=44.60664865854799}
输出> SensorReading{id='sensor7', timestamp=1637997843870, temperature=75.45476603031697}
输出> SensorReading{id='sensor1', timestamp=1637997843870, temperature=69.41033685906116}
--------------------------------------------------------------
输出> SensorReading{id='sensor4', timestamp=1637997844878, temperature=16.87517887988355}
输出> SensorReading{id='sensor5', timestamp=1637997844878, temperature=63.17579488504176}
输出> SensorReading{id='sensor2', timestamp=1637997844878, temperature=52.74636715490602}
输出> SensorReading{id='sensor3', timestamp=1637997844878, temperature=28.470195686766793}
输出> SensorReading{id='sensor10', timestamp=1637997844878, temperature=104.36050431699618}
输出> SensorReading{id='sensor8', timestamp=1637997844878, temperature=88.61411003722009}
输出> SensorReading{id='sensor9', timestamp=1637997844878, temperature=23.911634530531156}
输出> SensorReading{id='sensor6', timestamp=1637997844878, temperature=46.18522180961348}
输出> SensorReading{id='sensor7', timestamp=1637997844878, temperature=76.22117252989689}
输出> SensorReading{id='sensor1', timestamp=1637997844878, temperature=68.45890289027503}
...

注意的地方：
这里实现SourceFunction来完成自定义数据源的功能，其特点是不具有并行特性的，也就是说下面的代码在运行会报错。

 DataStream<SensorReading> objectDataStream = executionEnvironment.addSource(new MySensorSource()).setParallelism(2);

可以使用ParallelSourceFunction或RichParallelSourceFunction，他们俩都可以设置并行度，并且RichParallelSourceFunction还有额外的生命周期的方法。