1) mock data
2) DataFrame vs Dataset
DataFrame = Dataset[Row]
SchemaRDD ============> DataFrame ==========> Dataset
rename due to compile-time type safety(编译时就能提示错误)
OO structure change
Catalog API 对元数据的管理:库,表,列,cache cache有几种不同方式
SQLConf 最权威的API 比官网文档还权威
select /*+ mapjoin(A)*/
ThriftServer/HiveServer2 ==> Spark Application(SparkContext)
beeline
jdbc
*****
Hive on Spark
SparkTask
SQL==> MapTask/ReduceTask
==> SparkTask
2) DataFrame vs Dataset
DataFrame = Dataset[Row]
SchemaRDD ============> DataFrame ==========> Dataset
rename due to compile-time type safety(编译时就能提示错误)
OO structure change
Catalog API 对元数据的管理:库,表,列,cache cache有几种不同方式
SQLConf 最权威的API 比官网文档还权威
select /*+ mapjoin(A)*/
ThriftServer/HiveServer2 ==> Spark Application(SparkContext)
beeline
jdbc
*****
Hive on Spark
SparkTask
SQL==> MapTask/ReduceTask
==> SparkTask