java iterator 类型转换,类型不匹配：无法从Iterator< String>转换为在Java Spark中...

Exception in thread "main" java.lang.Error: Unresolved compilation problem:

Type mismatch: cannot convert from Iterator to Iterable

at com.spark.wordcount.lession1.WordCount2.main(WordCount2.java:26)

SparkConf conf = new SparkConf().setAppName("cust data").setMaster("local[*]");

JavaSparkContext sc = new JavaSparkContext(conf);

JavaRDD lines = sc.textFile("C:\\\\Users\\\\dell\\\\Desktop\\\\simple_text_file.txt");

JavaRDD words = lines.flatMap(s -> Arrays.asList(SPACE.split(s)).iterator());

JavaPairRDD ones = words.mapToPair(s -> new Tuple2<>(s, 1));

JavaPairRDD counts = ones.reduceByKey((i1, i2) -> i1 + i2);

List> output = counts.collect();

for (Tuple2,?> tuple : output) {

System.out.println(tuple._1() + ": " + tuple._2());

}

解决方案

You are mixing incompatible versions of Spark / code:

In Spark 1.x FlatMapFunction.call is Iterable call(T t).

You should either upgrade Spark dependency to 2.x and keep your current code or use FlatMapFunction compatible with 1.x branch:

JavaRDD words = lines.flatMap(new FlatMapFunction() {

@Override

public Iterable call(String s) {

return Arrays.asList(SPACE.split(s));

}

});