flink的transformation算子-keyBy

keyBy算子是将DataStream里的数据按照指定的key进行分组,具体实现可大致分为两类,1 传入的参数为下标值,这种一般只适用于DataStream里的数据类型为元组方可使用,2 传入的参数为keySelector或者其引用(java的lambda表达式),实现代码如下:

import org.apache.flink.api.common.operators.Keys;
import org.apache.flink.api.java.functions.KeySelector;
import org.apache.flink.api.java.tuple.Tuple2;
import org.apache.flink.streaming.api.datastream.DataStreamSource;
import org.apache.flink.streaming.api.datastream.KeyedStream;
import org.apache.flink.streaming.api.datastream.SingleOutputStreamOperator;
import org.apache.flink.streaming.api.environment.StreamExecutionEnvironment;


public class KeyByDemo4 {

    public static void main(String[] args) throws Exception {


        StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();

        //辽宁省,沈阳市,1000
        //辽宁省,大连市,2000
        //山东省,青岛市,3000
        //山东省,烟台市,2000
        //辽宁省,大连市,2000
        DataStreamSource<String> lines = env.socketTextStream("localhost", 8888);

        SingleOutputStreamOperator<ProvinceCityAndMoney2> tpDataStream = lines.map(line -> {
            String[] fields = line.split(",");
            return ProvinceCityAndMoney2.of(fields[0], fields[1], Double.parseDouble(fields[2]));
        });

        //KeyedStream<ProvinceCityAndMoney, Tuple> grouped = tpDataStream.keyBy("province", "city");

//        KeyedStream<ProvinceCityAndMoney, Tuple2<String, String>> grouped = tpDataStream.keyBy(new KeySelector<ProvinceCityAndMoney, Tuple2<String, String>>() {
//            @Override
//            public Tuple2<String, String> getKey(ProvinceCityAndMoney value) throws Exception {
//                return Tuple2.of(value.getProvince(), value.getCity());
//            }
//        });

        KeyedStream<ProvinceCityAndMoney2, Tuple2<String, String>> grouped = tpDataStream.keyBy(new KeySelector<ProvinceCityAndMoney2, Tuple2<String, String>>() {
            @Override
            public Tuple2<String, String> getKey(ProvinceCityAndMoney2 value) throws Exception {
                return Tuple2.of(value.province, value.city);
            }
        });

        SingleOutputStreamOperator<ProvinceCityAndMoney2> summed = grouped.sum("money");

        summed.print();

        env.execute();
    }

    //自定义的Pojo类封装数据,必须是Public
    public static class ProvinceCityAndMoney2 {

        public String province;
        public String city;
        public Double money;

        public ProvinceCityAndMoney2(){}

        public ProvinceCityAndMoney2(String province, String city, Double money) {
            this.province = province;
            this.city = city;
            this.money = money;
        }

        public static  ProvinceCityAndMoney2 of(String province, String city, Double money) {
            return new ProvinceCityAndMoney2(province, city, money);
        }

        @Override
        public String toString() {
            return "ProvinceCityAndMoney{" +
                    "province='" + province + '\'' +
                    ", city='" + city + '\'' +
                    ", money=" + money +
                    '}';
        }
    }
}

  • 3
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值