Flink常用Transformation算子(一)

在Flink学习中，会常用到一些转换的算子，其中比如map，flatMap，filter，reduce等，进行数据转换，本篇是文章的第一部分，主要介绍简单的转换

• map算子，即对原来数据进行一个数据的转换,下面的式子仅仅是对value进行转换，完全也可以返回一个tuple

DataStream<Tuple2<String,Integer>> wordcount = env.fromElements(
                new Tuple2<>("hello",1),
                new Tuple2<>("hello",2),
                new Tuple2<>("great",2));
        wordcount.map(new MapFunction<Tuple2<String, Integer>, Integer>() {
            @Override
            public Integer map(Tuple2<String,Integer> value) throws Exception{
                return value.f1 +1;
            }
        }).print();
env.execute("测试Map算子");

• reduce算子，对元素进行聚合，将输入的KeyedStream通过自定义的ReduceFunction进行聚合处理，自定义函数必须满足运算结合律和交换律，

DataStream<Tuple2<String,Integer>> wordcount = env.fromElements(
                new Tuple2<>("hello",1),
                new Tuple2<>("hello",2),
                new Tuple2<>("great",2));
        wordcount.keyBy(0).sum(1).print();
        wordcount.keyBy(0).reduce(
                new ReduceFunction<Tuple2<String, Integer>>() {
                    @Override
                    public Tuple2<String, Integer> reduce(Tuple2<String, Integer> t1, Tuple2<String, Integer> t2) throws Exception {
                        return Tuple2.of(t1.f0,t1.f1+t2.f1);
                    }
                }
        ).print();

• filter算子，顾名思义即对DataStream进行过滤，满足一定条件之后的数据才进入新的DataStream中，代码如下，用了lambada表达式

DataStream<Tuple2<String,Integer>> wordcount = env.fromElements(
        new Tuple2<>("hello",1),
        new Tuple2<>("hello",2),
        new Tuple2<>("great",2));
wordcount.filter((value) -> value.f1 % 2 ==0).print();

目前就写这些了，后续再丰富一下文档