Spark DataFrame中的join类型inner join, left join, right join, full join

Spark DataFrame中join与SQL很像，都有 inner join, left join, right join, full join

那么join方法如何实现不同的join类型呢？

看其原型

def join(right : DataFrame, usingColumns : Seq[String], joinType : String) : DataFrame
def join(right : DataFrame, joinExprs : Column, joinType : String) : DataFrame

可见，可以通过传入String类型的joinType来实现。

joinType可以是”inner”、“left”、“right”、“full”分别对应inner join, left join, right join, full join，默认值是”inner”，代表内连接

>>>personDataFrame.join(orderDataFrame, personDataFrame("id_person") === orderDataFrame("id_person")).show()

>>>personDataFrame.join(orderDataFrame, personDataFrame("id_person") === orderDataFrame("id_person"), "inner").show()

结果如下：

id_person    name    address    id_order    orderNum    id_person

1    张三    深圳    3    533    1

1    张三    深圳    4    444    1

2    李四    成都    1    325    2

3    王五    厦门    2    34    3

 

“left”,”left_outer”或者”leftouter”代表左连接

 

>>>personDataFrame.join(orderDataFrame, personDataFrame("id_person") === orderDataFrame("id_person"), "left").show()

>>>personDataFrame.join(orderData