Spark 累加器 Accumulator

Spark的累加器Accumulator中, Executor内只能做add操作， 只有Driver才能读取它的值。

    val doubleAcc: DoubleAccumulator = sc.doubleAccumulator("doubleAcc")
    val collAcc: CollectionAccumulator[Double] = sc.collectionAccumulator[Double]("collAcc")
    sc.parallelize(Array(1.1,-2.2,3.3)).foreach(x =>{
      doubleAcc.add(x)
      collAcc.add(x)
    })

    println(doubleAcc.value)
    println(collAcc.value)

转载于:https://my.oschina.net/dreamness/blog/3093829