3000门徒内部训练绝密视频（泄密版）第3课：Scala中函数式编程彻底精通及Spark源码阅读

Scala中函数式编程彻底精通及Spark源码阅读

函数可以不依赖于类，函数可以作为函数的参数，函数可以作为函数的返回值
=>表明对左面的参数进行右面的加工

函数赋值给变量需要在函数名后面加空格和下划线

scala> def fun1(name: String){println(name)}
fun1: (name: String)Unit

scala> val fun1_v = fun1 _
fun1_v: String => Unit = <function1>

scala> fun1("Spark")
Spark

scala> fun1_v("Spark")
Spark

scala> fun1_v("Scala")
Scala

scala> val fun2 = (content: String) => println(content)
fun2: String => Unit = <function1>
//匿名函数，不需要给函数命名的函数，一般会把匿名函数赋值给变量

scala> val hiScala = (content:String) => println(content)
hiScala: String => Unit = <function1>
scala> def bigData(func:(String) => Unit, content:String){func(content)}
bigData: (func: String => Unit, content: String)Unit
scala> bigData(hiScala,"Spark")
Spark

高阶函数，函数的参数也是函数

举例：map函数是传进来一个函数

scala> val array = Array(1,2,3,4,5,6,7,8,9)
array: Array[Int] = Array(1, 2, 3, 4, 5, 6, 7, 8, 9)

scala> array.map(item => 2*item)
res7: Array[Int] = Array(2, 4, 6, 8, 10, 12, 14, 16, 18)

scala> array.map(item => println(item))
1
2
3
4
5
6
7
8
9
res8: Array[Unit] = Array((), (), (), (), (), (), (), (), ())

高阶函数的返回值也可能是函数

scala> def func_Returned(content: String) = (message:String) => println(content+"  "+message)
func_Returned: (content: String)String => Unit

scala> func_Returned("Spark")
res10: String => Unit = <function1>

scala> val returned = func_Returned("Spark")
returned: String => Unit = <function1>

scala> returned("Scala")
Spark  Scala

高阶函数可以自动推断参数类型，对于只有一个参数的可以省略小括号，在函数的参数的作用的函数体内，只使用一次这个参数的具体的值，我们可以将参数名称省略用下划线代替

scala> def spark(func:(String) => Unit,name: String){func(name)}
spark: (func: String => Unit, name: String)Unit

scala> spark((name: String) => println(name), "Scala")
Scala
--自己写的关于map：map((item: Int) => 2*item)

scala> spark((name) => println(name), "Scala")
Scala
--自己写的关于map：map((item) => 2*item)

scala> spark(name => println(name), "Scala")
Scala
--自己写的关于map：map(item => 2*item)

scala> spark(println(_),"Scala")
Scala

scala> spark(println,"Scala")
Scala

scala> array.map(2*_)
res17: Array[Int] = Array(2, 4, 6, 8, 10, 12, 14, 16, 18)

scala> array.map(2*_).foreach(println(_))
2
4
6
8
10
12
14
16
18

scala> array.map(2*_).foreach(println _)
2
4
6
8
10
12
14
16
18
scala> array.map(2*_).foreach(println)
2
4
6
8
10
12
14
16
18


scala> array.map(2*_).filter(_ > 10 ).foreach(println)
12
14
16
18

闭包，函数的内部变量超出函数有效作用域，内部变量依旧可以被外部访问

scala> def scala(content:String) = (message : String) => println(content + " : " + message)
scala: (content: String)String => Unit

scala> val func_Result = scala("Spark")
func_Result: String => Unit = <function1>

scala> func_Result("Flink")
Spark : Flink

闭包内部：Scala内部生成了一个对象，将content作为一个成员

Single Abstract Method;

Currying:一个函数有两个参数可以把他转换成2个函数，第一个函数接收原先的第一个参数，第二个参数接收原先函数的第二个参数。

scala> def sum(x: Int, y: Int) = x + y
sum: (x: Int, y: Int)Int

scala> sum(3,4)
res29: Int = 7

scala> def sum_Currying(x : Int) = (y : Int) => x + y 
sum_Currying: (x: Int)Int => Int

scala> sum_Currying(1)(2)
res31: Int = 3

scala> def sum_Currying_Better(x : Int) (y : Int) = x + y 
sum_Currying_Better: (x: Int)(y: Int)Int

scala> sum_Currying_Better(1)(3)
res33: Int = 4

reduceLeft,从左至右

scala> (1 to 3).reduceLeft(_-_)
res41: Int = -4

几个重要的集合函数：

scala> val list = List("Scala","Spark")
list: List[String] = List(Scala, Spark)

scala> list.map("The content is : " + _)
res42: List[String] = List(The content is : Scala, The content is : Spark)

scala> list.map(println)
Scala
Spark
res43: List[Unit] = List((), ())

scala> val cal = list.map("The content is : " + _)
cal: List[String] = List(The content is : Scala, The content is : Spark)

scala> cal
res44: List[String] = List(The content is : Scala, The content is : Spark)

scala> cal.flatMap(_.split(" "))
res46: List[String] = List(The, content, is, :, Scala, The, content, is, :, Spark)

scala> cal.flatMap(_.split(" ")).foreach(print)
Thecontentis:ScalaThecontentis:Spark

scala> list.zip(List(6,16,5))
res48: List[(String, Int)] = List((Scala,6), (Spark,16))

iterable接口(trait) 集合：immutable不可变，mutable可变

作业：统计一个文件夹下面所有文件的单词出现的总次数

import scala.io.Source.fromFile
import java.io.File
import scala.collection.mutable.LinkedHashMap
val myMap = LinkedHashMap[String,Int]()
new File("/home/tom/scalatest").listFiles.filter(_.isFile).map(fromFile).flatMap(_.getLines).flatMap(_.split(" ")).foreach((str:String) =>{ myMap.put(str, myMap.getOrElse(str,0)+1) })
myMap.foreach(println)