spark-sql建表语句限制_Spark SQL不允许我创建表，抱怨默认Metastore目录

Installed Spark 1.5 spark-1.5.0-bin-hadoop2.6 on my local machine.

Ran $ ./bin/spark-shell

Tried, following the doc to create a table, getting this:

> SQL context available as sqlContext.

>

> scala> sqlContext.sql("CREATE TABLE IF NOT EXISTS src (key INT, value

> STRING)"); 15/09/22 22:18:13 ERROR DDLTask:

> org.apache.hadoop.hive.ql.metadata.HiveException:

> MetaException(message:file:/user/hive/warehouse/src is not a directory

> or unable to create one) at

> org.apache.hadoop.hive.ql.metadata.Hive.createTable(Hive.java:720)

Tried passing the hive parameter for this, but didnt work:

> $ ./bin/spark-shell --conf hive.metastore.warehouse.dir=./ Warning:

> Ignoring non-spark config property: hive.metastore.warehouse.dir=./

Finally tried the CLI itself, but getting the same issue.

Where do i change the hive warehouse parameter location ? I dont have Hadoop installed at the moment, nor hive.

thanks,

Matt

解决方案

Metadata of hive tables are strored in metastore, hive context adds support for finding tables in the MetaStore.

import org.apache.spark.sql.hive.HiveContext

val hiveContext = new HiveContext(sc)

val myDF = sql("select * from mytable")

You will get dataFrame as result

myDF: org.apache.spark.sql.DataFrame = [.....]