scala中的排序sortBy和map例子

API接口点击

一直不会用sortBy，都是用sortWith,今天偶然看到一个sortBy的写法，觉得很高级，分享一下

var ll = List[(String, Int, Int)](("a",1, 400),("b",3, 600),("m",3, 100),("c",2, 40))
println(ll)

var a  = ll.map{case Tuple3(name:String, age:Int, salary:Int)=>(name,age, salary)}.sortBy{case(_,age,_)=>(age)}
println("a==",a)
var aa  = ll.map{case Tuple3(name:String, age:Int, salary:Int)=>(name,age, salary)}.sortBy{case(_,age,_)=>(-age)}
println("aa==",aa)
var aaa = ll.map{case Tuple3(name:String, age:Int, salary:Int)=>(name,age, salary)}.sortBy{case(_,age,salary)=>(age, salary)}
println("aaa==",aaa)

var b = ll.map{case Tuple3(name:String, age:Int, salary:Int)=>(name,age, salary)}.sortBy{case(_,age,salary)=>(age, salary)}.map{case (name,_,_) => name}

println("bb==",b)
var bb = ll.map{case Tuple3(name:String, age:Int, salary:Int)=>(name,age, salary)}.sortBy{case(_,age,salary)=>(age, salary)}
  .map{case (name,_,salary) => (name,salary)}

println("bb==",bb)

将Tuple中的3个变量分别取名为 name age 和salary

ll.map{case Tuple3(name:String, age:Int, salary:Int)=>(name,age, salary)}

利用某个字段排序，默认正序，从小到大，其中case中变量必须与总变量数量相同，如果不需要，可用"_"省略，但不能不写

.sortBy{case(,age,)=>(age)}

倒序只需要在age前加"-"

.sortBy{case(,age,)=>(-age)}

再用map筛选掉不需要的字段

.map{case (name,,) => name}

参考链接：Scala – 通过第一个元素反向排序元组 | scala神奇的sortBy方法

原例子：来源于 王吉 SparrowRecSyc

  def processItemSequence(sparkSession: SparkSession, rawSampleDataPath: String): RDD[Seq[String]] ={

    //path of rating data
    val ratingsResourcesPath = this.getClass.getResource(rawSampleDataPath)
    val ratingSamples = sparkSession.read.format("csv").option("header", "true").load(ratingsResourcesPath.getPath)

    //sort by timestamp udf
    val sortUdf: UserDefinedFunction = udf((rows: Seq[Row]) => {
      rows.map { case Row(movieId: String, timestamp: String) => (movieId, timestamp) }
        .sortBy { case (_, timestamp) => timestamp }
        .map { case (movieId, _) => movieId }
    })

    ratingSamples.printSchema()

    //process rating data then generate rating movie sequence data
    val userSeq = ratingSamples
      .where(col("rating") >= 3.5)
      .groupBy("userId")
      .agg(sortUdf(collect_list(struct("movieId", "timestamp"))) as "movieIds")
      .withColumn("movieIdStr", array_join(col("movieIds"), " "))

    userSeq.select("userId", "movieIdStr").show(10, truncate = false)
    userSeq.select("movieIdStr").rdd.map(r => r.getAs[String]("movieIdStr").split(" ").toSeq)
  }