简单记录Flink批处理数据的一些算子，distinct，filter

1、读取原始文件csv

ExecutionEnvironment env = ExecutionEnvironment.getExecutionEnvironment();
DataSource<String> csvSource = env.readTextFile("文件路径", "UTF-8");

2、根据某一字段去重

//根据ID去重
DistinctOperator<Row> source = csvSource2.distinct(new KeySelector<Row, String>() {
        @Override
        public String getKey(Row value) {return (String)value.getField(0);}
    }
);

3、根据特定字段过滤

//只提取数据中第二列性别中为女的数据
FliterOperator<Row> source = csvSource.filter(new FilterFunction<String>() {
        @Override
        public boolean filter(String value) throws Exception {
            return (String)value.getField(1).equals("女");
        }
    }
);