[Scala进阶]-- Spark累加器（单例模式应用）

一、举例创建单例模式的累加器

import org.apache.spark.{Accumulator, SparkContext}

/**
  * Created by root on 20170618.
  * Update date:
  * Time: 12:04 PM
  *
  */
object SuccessRecordCounter {
  @volatile private var instance: Accumulator[Long] = null //共享变量

  def getInstance(sc: SparkContext): Accumulator[Long] = {
    if (instance == null) {
      synchronized {
        if (instance == null) {
          instance = sc.accumulator(0L, "SuccessRecordCounter")
        }
      }
    }
    instance
  }
}

 

 

二、使用

val successCounter = SuccessRecordCounter.getInstance(rdd.sparkContext)
successCounter.a