spark-DataFrame,DataSet

 

首先先看一个rdd,DF,DS的转换图

DataFrame创建：

//读取json文件  
scala> val df=spark.read.json("G:/ccData/person.json")
df: org.apache.spark.sql.DataFrame = [age: bigint, id: bigint ... 1 more field]

scala> df.show
+---+---+-------+
|age| id|   name|
+---+---+-------+
| 18|  1|     cc|
| 19|  2|   qiqi|
| 17|  3|xiaohei|
+---+---+-------+

//创建临时表 ，命名为person
scala> df.createTempView("person")

scala> spark.sql("select * from person").show
+---+---+-------+
|age| id|   name|
+---+---+-------+
| 18|  1|     cc|
| 19|  2|   qiqi|
| 17|  3|xiaohei|
+---+