hive show databases报错_生产Spark+Hive会遇见的拦路虎

问题不难，关键在于排查思路

1.抛错: Unable to instantiate SparkSession with Hive support because Hive classes are not found.

需要先将hadoop的core-site.xml，hive的hive-site.xml拷贝到project中1.1 测试代码

def main(args: Array[String]): Unit = {
    
    val spark: SparkSession = SparkSession
      .builder()
      .appName("www.ruozedata.com")
      .master("local[2]")
      .enableHiveSupport()
      .getOrCreate()
    val userClickDF = spark.table("user_click")
    userClickDF.show(10)
}

1.2 报错

Exception in thread "main" java.lang.IllegalArgumentException: Unable to instantiate SparkSession with Hive support because Hive classes are not found.
    at org.apache.spark.sql.SparkSession$Builder.enableHiveSupport(SparkSession.scala:869)
    at homework0522.OverwriteTopN$.main(OverwriteTopN.scala:12)
    at homework0522.OverwriteTopN.main(OverwriteTopN.scala)

1.3 查看源码