spark就应该这么玩

有需要的 

Spark DataFrame 写入 OrientDB

    val spark = SparkSession.builder().appName("SparkOrientDB").getOrCreate()
    import spark.implicits._
    import spark.sql

    // Vertex DataFrame
    spark.createDataFrame(List(
      ("a", "Alice", 34),
      ("b", "Bob", 36),
      ("c", "Charlie", 30),
      ("d", "David", 29),
      ("e", "Esther", 32),
      ("f", "Fanny", 36),
      ("g", "Gabby", 60)
    )).toDF("id", "name", "age")
      .write.format("org.apache.spark.orientdb.graphs")
      .option("dburl", "remote:localhost/graphdb")
      .option("user", "root")
      .option("password", "root")
      .option("vertextype", "Vgraphx")
      .mode("overwrite")
      .save()

    // Edge DataFrame
    spark.createDataFrame(List(
      ("a", "b", "friend"),
      ("b", "c", "follow"),
      ("c", "b", "follow"),
      ("f", "c", "follow"),
      ("e", "f", "follow"),
      ("e", "d", "friend"),
      ("d", "a", "friend"),
      ("a", "e", "friend")
    )).toDF("src", "dst", "relationship")
      .write.format("org.apache.spark.orientdb.graphs")
      .option("dburl", "remote:localhost/graphdb")
      .option("user", "root")
      .option("password", "root")
      .option("vertextype", "Vgraphx")
      .option("edgetype", "Egraphx")
      .mode("overwrite")
      .save()

• 1
• 2
• 3
• 4
• 5
• 6
• 7
• 8
• 9
• 10
• 11
• 12
• 13
• 14
• 15
• 16
• 17
• 18
• 19
• 20
• 21
• 22
• 23
• 24
• 25
• 26
• 27
• 28
• 29
• 30
• 31
• 32
• 33
• 34
• 35
• 36
• 37
• 38
• 39
• 40
• 41    可以联系我yxxy1717    qq 2137384986

单击 http://localhost:2480，查询 写入的顶点和边。如下图所示， 

Spark 读取 OrientDB 返回 DataFrame

    val vertices = spark.read
      .format("org.apache.spark.orientdb.graphs")
      .option("dburl", "remote:localhost/graphdb")
      .option("user", "root")
      .option("password", "root")
      .option("vertextype", "Vgraphx")
      .load()

    val edges = spark.read
      .format("org.apache.spark.orientdb.graphs")
      .option("dburl", "remote:localhost/graphdb")
      .option("user", "root")
      .option("password", "root")
      .option("edgetype", "Egraphx")
      .load()

    val g = GraphFrame(vertices, edges)

• 1
• 2
• 3
• 4
• 5
• 6
• 7
• 8
• 9
• 10
• 11
• 12
• 13
• 14
• 15
• 16
• 17

顶点输出如下，

g.vertices.show(false)

+-------+---+---+
|name   |id |age|
+-------+---+---+
|Bob    |b  |36 |
|David  |d  |29 |
|Charlie|c  |30 |
|Esther |e  |32 |
|Fanny  |f  |36 |
|Gabby  |g  |60 |
|Alice  |a  |34 |
+-------+---+---+

• 1
• 2
• 3
• 4
• 5
• 6
• 7
• 8
• 9
• 10
• 11
• 12
• 13

边的输出如下，

g.edges.show(false)

+---+------------+---+
|dst|relationship|src|
+---+------------+---+
|c  |follow      |b  |
|b  |follow      |c  |
|f  |follow      |e  |
|a  |friend      |d  |
|c  |follow      |f  |
|d  |friend      |e  |
|b  |friend      |a  |
|e  |friend      |a  |
+---+------------+---+