背景
graphx实现k-core比较简单,参考淘宝技术部之前的文章,已经给出了一个代码片段,基本上改改就可以定制自己的需求了。
code
import org.apache.spark._
import org.apache.spark.graphx._
import org.apache.spark.storage.StorageLevel
import org.apache.spark.graphx.lib._
// load the graph
val friendsGraph = GraphLoader.edgeListFile(sc, "data/friends.txt.", false, 512, StorageLevel.MEMORY_ONLY, StorageLevel.DISK_ONLY)
var degreeGraph = friendsGraph.outerJoinVertices(friendsGraph.degrees) {
(vid, vd, degree) &