Spark求平均数,父子关系

最新推荐文章于 2021-10-22 15:49:41 发布

青山流水在深谷

最新推荐文章于 2021-10-22 15:49:41 发布

阅读量973

点赞数

分类专栏： scala

本文链接：https://blog.csdn.net/dymkkj/article/details/78042106

版权

scala 专栏收录该内容

5 篇文章 0 订阅

订阅专栏

import org.apache.spark.{SparkConf, SparkContext}
object App extends App{


     val  conf = new SparkConf().setMaster("local[*]").setAppName("Jack")
     val  sc = new SparkContext(conf)


     val  arr = Array(("a",1),("a",1),("b",10),("b",3),("c",6))
     val  rdd = sc.makeRDD(arr);
     val  rdd2 = rdd.groupByKey()
     rdd2.foreach(x=>{

        val sum = x._2.sum.toDouble
        val amout = x._2.size.toDouble
        print("("+x._1+","+sum/amout+")")
    })


  }


爷孙关系:如A:B  B:C   A,C 是爷孙关系

object GeneratePS {
  def main(args: Array[String]): Unit = {
    val  conf = new SparkConf().setMaster("local[*]").setAppName("AF")
    val  sc = new SparkContext(conf)
    val  arr = Array(("A","B"),("B","C"),("D","E"),("E","F"),("H","M"))
    val  rdd = sc.makeRDD(arr).collect();
    for(i<- rdd){
      for(j<- rdd){
        if(i._2 == j._1){
          print("( P: "+i._1+", S: "+j._2+" )")
        }
      }
    }

  }