# scala map reduce 思想

## 1. map

val v = Vector(1, 2, 3, 4)
val v2 = v.map(n => n * 2)

scala> val v = Vector(1, 2, 3, 4)
v: scala.collection.immutable.Vector[Int] = Vector(1, 2, 3, 4)

scala> val v2 = v.map(n => n * 2)
v2: scala.collection.immutable.Vector[Int] = Vector(2, 4, 6, 8)

map这个单词在这里不是地图的意思，而是映射、关联，将源容器中的元素经过函数处理后一一映射到新容器中。上面代码中执行的函数就是乘以2的匿名函数，我们可以自己定义成其他函数，加2啊，乘5啊，平方啊，等等。

## 2. reduce

val v = Vector(1, 2, 3, 4)
val v3 = v.reduce((sum, n) => sum + n)

## 3. 具体例子

1 54
2 69
3 66
4 33
5 18
6 51
7 82
8 26
9 1

import org.apache.spark.SparkConf
import org.apache.spark.SparkContext

object AvgAgeCalculator {
def main(args: Array[String]): Unit = {
val conf = new SparkConf().setAppName("Spark Exercise:Average Age Calculator")
val sc = new SparkContext(conf)
val dataFile = sc.textFile("file:///Users/walle/Documents/spark_projects/sparkage/sample_age_data.txt", 5);

val count = dataFile.count()
//文件是对一行处理，这里对空格进行分割得到第二个，scala数组中是用()根据下标取元素
val ageData = dataFile.map(line => line.split(" ")(1))
//求和
val totalAge = ageData.map(age => Integer.parseInt( String.valueOf(age))).collect().reduce((a, b) => a + b)
val avgAge : Double = totalAge.toDouble / count.toDouble
println("Average Age is " + avgAge)
}

}

http://www.codeblogbt.com/archives/148029

©️2019 CSDN 皮肤主题: 大白 设计师: CSDN官方博客