一直不会用sortBy,都是用sortWith,今天偶然看到一个sortBy的写法,觉得很高级,分享一下
var ll = List[(String, Int, Int)](("a",1, 400),("b",3, 600),("m",3, 100),("c",2, 40))
println(ll)
var a = ll.map{case Tuple3(name:String, age:Int, salary:Int)=>(name,age, salary)}.sortBy{case(_,age,_)=>(age)}
println("a==",a)
var aa = ll.map{case Tuple3(name:String, age:Int, salary:Int)=>(name,age, salary)}.sortBy{case(_,age,_)=>(-age)}
println("aa==",aa)
var aaa = ll.map{case Tuple3(name:String, age:Int, salary:Int)=>(name,age, salary)}.sortBy{case(_,age,salary)=>(age, salary)}
println("aaa==",aaa)
var b = ll.map{case Tuple3(name:String, age:Int, salary:Int)=>(name,age, salary)}.sortBy{case(_,age,salary)=>(age, salary)}.map{case (name,_,_) => name}
println("bb==",b)
var bb = ll.map{case Tuple3(name:String, age:Int, salary:Int)=>(name,age, salary)}.sortBy{case(_,age,salary)=>(age, salary)}
.map{case (name,_,salary) => (name,salary)}
println("bb==",bb)
将Tuple中的3个变量分别取名为 name age 和salary
ll.map{case Tuple3(name:String, age:Int, salary:Int)=>(name,age, salary)}
利用某个字段排序,默认正序,从小到大,其中case中变量必须与总变量数量相同,如果不需要,可用"_"省略,但不能不写
.sortBy{case(,age,)=>(age)}
倒序只需要在age前加"-"
.sortBy{case(,age,)=>(-age)}
再用map筛选掉不需要的字段
.map{case (name,,) => name}
参考链接:Scala – 通过第一个元素反向排序元组 | scala神奇的sortBy方法
原例子:来源于 王吉 SparrowRecSyc
def processItemSequence(sparkSession: SparkSession, rawSampleDataPath: String): RDD[Seq[String]] ={
//path of rating data
val ratingsResourcesPath = this.getClass.getResource(rawSampleDataPath)
val ratingSamples = sparkSession.read.format("csv").option("header", "true").load(ratingsResourcesPath.getPath)
//sort by timestamp udf
val sortUdf: UserDefinedFunction = udf((rows: Seq[Row]) => {
rows.map { case Row(movieId: String, timestamp: String) => (movieId, timestamp) }
.sortBy { case (_, timestamp) => timestamp }
.map { case (movieId, _) => movieId }
})
ratingSamples.printSchema()
//process rating data then generate rating movie sequence data
val userSeq = ratingSamples
.where(col("rating") >= 3.5)
.groupBy("userId")
.agg(sortUdf(collect_list(struct("movieId", "timestamp"))) as "movieIds")
.withColumn("movieIdStr", array_join(col("movieIds"), " "))
userSeq.select("userId", "movieIdStr").show(10, truncate = false)
userSeq.select("movieIdStr").rdd.map(r => r.getAs[String]("movieIdStr").split(" ").toSeq)
}