Spark之Scala学习之路（三）

函数式编程:
函数定义：

def fun1(name:String){println(name)}
val fun1_v=fun1 _    //名称 空格 下划线
//fun1_v: String => Unit = <function1> 左边变量进行右边的加工
fun1(1)
fun1_v    //调用方法

匿名函数使用 （无名称）

val fun2=(content:String) =>println(content)
val hiScala=(content:String) =>println(content)

def bigData(func:(String) =>Unit,content:String){func(content)}
bigData(hiScala,"spark")  //spark
//传入一个函数进来，要求输入是字符串，返回类型是Unit
val array=Array(1,2,3,4,5,6,7,8,9)
array.map(item => 2*item )

函数的返回值是函数

def func_Returned(content:String)=(message:String) => println(message)
func_Returned("spark")

def func_Returned(content:String)=(message:String) => println(content+"   "+message)
val returned=func_Returned("Spark")
returned("Scala")  //Spark  Scala 

简化：如果在函数的函数体内只使用一次函数的输入的参数值 参数的名称省略掉用下划线代替

def spark(func:(content:String)=>Unit,name:String){func(name)}
def spark(func:(String)=>Unit,name:String){func(name)}
spark((name:String) => println(name),"Scala")  //以下为简化方法
spark((name) => println(name),"Scala")
spark(name => println(name),"Scala")
spark(println(_),"Scala")
spark(println,"Scala")

array.map(2*_)
array.map(2*_).foreach(println)
array.map(2*_).filter(_>10).foreach(println)

闭包：函数变量超出有效作用域，还能对函数内部进行访问

def scala(content:String) =(message:String)=>println(content+":"+message)
val funcResult=scala("Spark")
funcResult("Flink")

柯里化

def sum(x:Int,y:Int) =x+y
sum(1,2)
def sum_Curring(x:Int) = (y:Int) =>x+y
sum_Curring(1)(2)
def sum_Curring_Butter(x:Int)(y:Int) =x+y
sum_Curring_Butter(1)(2)

(1 to 100).reduceLeft(_+_)
val list=List("scala","spark","hadoop")
val cal=list.map("the content is" +_)
cal.flatMap(_.split(" "))

list.zip(List(10,6,5)) //List[(String, Int)] = List((scala,10), (spark,6), (hadoop,5))

统计对文件夹中所有文本的单词个数

import scala.io.Source._
import scala.io.Source
import java.io._
object WordCount {
  var map = Map.empty[String, Int]

  def main(args: Array[String]): Unit = {
    try{
      Make(new File("E:/test"))
    }catch {
       case _:FileNotFoundException=>println("error")
    }
    map.foreach(println)
  }

  def Make(file: File) = {
    file.listFiles.foreach { file =>
      if (file.isFile()) {
        txtFile(file)
        println(file)
      }
    }
  }

  def txtFile(file: File) = {
    lazy val content = Source.fromFile(file)
    for (line <- content.getLines()) {
      count(line)
    }
  }

  def count(line: String) = {
    for (word <- line.split(" ")) {
      if (map.contains(word))
        map += (word -> (map(word) + 1))
      else
        map += (word -> 1)
    }
  }

}