HIVE的错题本

一， 初始化元数据出错

使用derby作为元数据库，初始化时失败。

/export/server/hive-2.1.0/bin/schematool -dbType derby -initSchema

0: jdbc:derby:> CREATE FUNCTION "APP"."NUCLEUS_ASCII" (C CHAR(1)) RETURNS INTEGER LANGUAGE JAVA PARAMETER STYLE JAVA READS SQL DATA CALLED ON NULL INPUT EXTERNAL NAME 'org.datanucleus.store.rdbms.adapter.DerbySQLFunction.ascii' 
Error: FUNCTION 'NUCLEUS_ASCII' already exists. (state=X0Y68,code=30000)

Closing: 0: jdbc:derby:;databaseName=metastore_db;create=true
org.apache.hadoop.hive.metastore.HiveMetaException: Schema initialization FAILED! Metastore state would be inconsistent !!
Underlying cause: java.io.IOException : Schema script failed, errorcode 2
org.apache.hadoop.hive.metastore.HiveMetaException: Schema initialization FAILED! Metastore state would be inconsistent !!
	at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:291)
	at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:264)
	at org.apache.hive.beeline.HiveSchemaTool.main(HiveSchemaTool.java:505)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
	at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.io.IOException: Schema script failed, errorcode 2
	at org.apache.hive.beeline.HiveSchemaTool.runBeeLine(HiveSchemaTool.java:390)
	at org.apache.hive.beeline.HiveSchemaTool.runBeeLine(HiveSchemaTool.java:347)
	at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:287)

原因：已经执行过mysql的初始化，使用derby导致冲突，将以前的数据库改名可以解决这个问题。

cd  /export/server/hive-2.1.0/scripts/metastore/upgrade/derby

mv metastore_db metastore_db.tmp

然后重新初始化，即可成功。

二，FAILED: SemanticException Line 0:-1 Partition not found

这种情况一般是你创建的表是多级分区（比如year，month)，若操作时指定一个分区是不可以的(只加day或者hour)

insert overwrite table test partition (
    year=2021
    month='17'
)

三，ob Submission failed with exception 'java.io.IOException(org.apache.hadoop.yarn.exceptions.YarnException: Failed to submit application_1621563516281_0049 to YARN : Application application_1621563516281_0049 submitted by user root to unknown queue: ‘prod’

设置队列名称时不要用引号：

#正确
set mapreduce.job.queuename=prod;
#错误
set mapreduce.job.queuename='prod';

四，Could not connect to hadoop02:10000 (code THRIFTTRANSPORT): TTransportException(‘Could not connect to hadoop02:10000’,)

由于 HiveServer2 的 Java 堆栈大小（字节）过小，导致意外退出

参考文献

五，Hive执行sql语句，提交了之后一直卡住

MapReduce的内存分配

查看Yarn，发现是UNDEFINED状态：

再点进去查看日志，发现资源不足：

Resource request: <memory:1024, vCores:1> exceeds maximum AM resource allowed

查看NodeManager的配置，发现 yarn.nodemanager.resource.memory-mb=1G，nodemanager只配了1G的内存，表示该节点上YARN可使用的物理内存总量，而Map任务需要1G的内存，自然不够了。

解决办法就是扩大这个配置的值。

但这个值和yarn.scheduler.minimum-allocation-mb以及yarn.scheduler.maximum-allocation-mb有关系，必须要介于二者直接。

这两个值在yarn的web ui上可以看到：

六，MapReduce任务卡住不动

ACCEPTED: waiting for AM container to be allocated, launched and register with RM.

应该也是内存问题，见上一个问题的第一种解法。