spark 完成二次排序

先根据第一列的数字从大到小排序,如果相等,在按照第二列进行排序.
文件内容:
file3.txt

23 4
23 5
2 4
2 5
6 7
6 8
10 23
10 23
43 45
43 45
46 56
46 56

排序代码:
1、这个一个类:SecondarySortkey

/**
  * @author 
  * @date 2020-09-09 15:02
  * @version 1.0
  */
class SecondarySortkey(val first:Int,val second:Int) extends Ordered [SecondarySortkey] with Serializable {
   def compare(other: SecondarySortkey): Int = {
     if (this.first - other.first!=0) {
       this.first -other.first
     } else {
       this.second - other.second
     }
   }
}

2、这是一个对象

/**
  * @author navy
  * @date 2020-09-09 17:44
  * @version 1.0
  */
import org.apache.spark.SparkConf
import org.apache.spark.SparkContext
object SecondarySortApp {
  def main(args: Array[String]) = {
    val conf = new SparkConf().setAppName("SecondarySortApp!").setMaster("local")
    val sc = new SparkContext(conf)
    val lines=sc.textFile("/Users/navyliu/Downloads/spark/sparkcase/file3.txt",1)
    val pairWithSortKey = lines.map(line=>(new SecondarySortkey(line.split(" ")(0).toInt,line.split(" ")(1).toInt),line))
    val sorted = pairWithSortKey.sortByKey(false)
    val sortedResult = sorted.map(sortedLine=>sortedLine._2)
    sortedResult.collect().foreach(println)
  }
}

执行完之后,可以看到排序的结果

欲了解更多scala编程的内容,请关注公众号: