Spark map与flatmap区别

总结：

- Spark 中 map函数会对每一条输入进行指定的操作，然后为每一条输入返回一个对象；

- 而flatMap函数则是两个操作的集合——正是“先映射后扁平化”：

   操作1：同map函数一样：对每一条输入进行指定的操作，然后为每一条输入返回一个对象

   操作2：最后将所有对象合并为一个对象

 

val rdd1 = sc.parallelize(Seq(("one two three four five six seven"), ("one two three four five six seven"), ("one two three four five six seven")))
 
 
rdd1.map(_.split(" ")).collect
/*
res6: Array[Array[String]] = Array(Array(one, two, three, four, five, six, seven),
                                  Array(one, two, three, four, five, six, seven),
                                  Array(one, two, three, four, five, six, seven))
*/
 
rdd1.flatMap(_.split(" ")).collect
/*
res7: Array[String] = Array(one, two, three, four, five, six, seven,
                            one, two, three, four, five, six, seven,
                            one, two, three, four, five, six, seven)
*/