Flink常用算子及其使用

Flink常用算子及其使用

常用算子：

基本使用：
1、map

        //创建执行环境(批处理)
        ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
        //map算子
        DataSource<Integer> data1 = env.fromCollection(Arrays.asList(1, 2, 3, 4, 5));
        data1.map(new MapFunction<Integer, Integer>() {
            @Override
            public Integer map(Integer integer) throws Exception {
                return integer+1;
            }
        }).print();

输出结果：

flatMap

        //flatMap:
        DataSource<String> data2 = env.fromCollection(Arrays.asList("hello", "my name is xiaobai", "who are you"));
        data2.flatMap(new FlatMapIterator<String, String>() {
            @Override
            public Iterator<String> flatMap(String s) throws Exception {
                return Arrays.asList(s.split(" ")).iterator();
            }
        }).print();

结果：

filter:

        //filter过滤
        DataSource<Integer> data3 = env.fromCollection(Arrays.asList(1, 2, 3, 4, 5));
        data3.filter(new FilterFunction<Integer>() {
            @Override
            public boolean filter(Integer integer) throws Exception {
                return integer <= 3;
            }
        }).print();

结果：

reduce
对每两个数据(及其结果)连续操作

        //reduce
       DataSource<Integer> data3 = env.fromCollection(Arrays.asList(1, 2, 3, 4, 5));
        data3.reduce(new ReduceFunction<Integer>() {
            @Override
            public Integer reduce(Integer integer, Integer t1) throws Exception {
               return integer+t1;
            }
        }).print();

结果：