sparkSession访问hive，PARTITIONS表不可用

最新推荐文章于 2022-05-05 12:41:03 发布

猩猩不相惜

最新推荐文章于 2022-05-05 12:41:03 发布

阅读量392

点赞数

分类专栏：大数据 spark hive

本文链接：https://blog.csdn.net/weixin_42737797/article/details/99651102

版权

大数据同时被 3 个专栏收录

2 篇文章 0 订阅

订阅专栏

spark

1 篇文章 0 订阅

订阅专栏

hive

1 篇文章 0 订阅

订阅专栏

为分布式搭建hive on spark;

记录耗时最久的一个问题：太气人了

2019-08-15 18:53:03 ERROR Datastore:115 - Error thrown executing ALTER TABLE `PARTITIONS` ADD COLUMN `TBL_ID` BIGINT NULL : Table 'hive.PARTITIONS' doesn't exist
java.sql.SQLSyntaxErrorException: Table 'hive.PARTITIONS' doesn't exist

-----------------------------

2019-08-15 18:53:04 ERROR ConnectionHandle:388 - Database access problem. Killing off this connection and all remaining connections in the connection pool. SQL State = HY000
2019-08-15 18:53:04 ERROR Datastore:115 - Error thrown executing ALTER TABLE `PARTITIONS` ADD COLUMN `TBL_ID` BIGINT NULL : Table 'hive.PARTITIONS' doesn't exist
java.sql.SQLSyntaxErrorException: Table 'hive.PARTITIONS' doesn't exist

-------------------------------

--------------------------------------

2019-08-15 18:53:05 ERROR ConnectionHandle:388 - Database access problem. Killing off this connection and all remaining connections in the connection pool. SQL State = HY000
2019-08-15 18:53:05 WARN Query:106 - Query for candidates of org.apache.hadoop.hive.metastore.model.MPartitionColumnStatistics and subclasses resulted in no possible candidates
Schema Transaction threw exception "Add classes to Catalog "hive", Schema """
org.datanucleus.exceptions.NucleusDataStoreException: Schema Transaction threw exception "Add classes to Catalog "hive", Schema """

Caused by: java.sql.SQLException: Connection is closed!
   at com.jolbox.bonecp.ConnectionHandle.checkClosed(ConnectionHandle.java:459)
   at com.jolbox.bonecp.ConnectionHandle.getAutoCommit(ConnectionHandle.java:776)
   at org.datanucleus.store.rdbms.AbstractSchemaTransaction.execute(AbstractSchemaTransaction.java:131)
   ... 122 more
Nested Throwables StackTrace:
java.sql.SQLException: Connection is closed!

-----------------------

以上是报错信息-------太糊人

很明显是mysql的问题，过程太愚蠢就不写了，直接上结论：

高版本的mysql.driver的jar包不兼容一些就的sql脚本，导致创建表执行失败，

解决办法：

将driver的jar，换回低版本，我换回了5.1的jar。

注解：本来应用的是与mysql完全对应的driver，就一直没有想到真的是它的问题。郑重警告有类似洁癖的猩猩们。