//spark 读取csv文件
public static class ParseLine implements FlatMapFunction<Tuple2 <String,String>,String []> {
public Iterable<String[]> call(Tuple2<String, String> file) throws Exception{
CSVReader csvReader = new CSVReader(new StringReader(file._2()));
return csvReader.readAll();
}
}
SparkConf sc = new SparkConf();
DataFrame data = sc.wholetextFile();
RDD<Object> keyedRDD = data.flatMap(new ParseLine());
Spark+JAVA+读取CSV文件
最新推荐文章于 2024-01-11 10:24:15 发布