spark算子底层实现
Transformation算子(不触发Action)不产生shufflemap 底层调用MapPartitionsRDD --TaskContext 获取上下文new MapPartitionsRDD[U, T](this, (_, _, iter) => iter.map(cleanF))//(TaskContext, partition index, iterator)val list = sc.parallelize(List(1, 2, 3, 4, 5, 6, 7), 2).
原创
2020-08-26 18:06:41 ·
310 阅读 ·
0 评论