Hive安装错误总结

错误1：Exception in thread “main” java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient

详细信息：

Fri May 18 09:51:14 CST 2018 WARN: Establishing SSL connection without server's identity verification is not recommended.According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
Exception in thread "main" java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
        at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:614)
        at org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:549)
        at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:750)
        at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:686)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
        at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:232)
        at org.apache.hadoop.hive.ql.metadata.Hive.<init>(Hive.java:384)
        at org.apache.hadoop.hive.ql.metadata.Hive.create(Hive.java:328)
        at org.apache.hadoop.hive.ql.metadata.Hive.getInternal(Hive.java:308)
        at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:284)
        at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:581)
        ... 9 more
Caused by: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClien             t
        at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1654)
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.<init>(RetryingMetaStoreClient.java:83)
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:133)
        at org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104)
        at org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3496)
        at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3548)
        at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3528)
        at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3790)
        at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:244)
        at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:227)
        ... 14 more
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1652)
        ... 23 more
Caused by: MetaException(message:Version information not found in metastore. )
        at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:83)
        at org.apache.hadoop.hive.metastore.RetryingHMSHandler.getProxy(RetryingHMSHandler.java:92)
        at org.apache.hadoop.hive.metastore.HiveMetaStore.newRetryingHMSHandler(HiveMetaStore.java:6608)
        at org.apache.hadoop.hive.metastore.HiveMetaStoreClient.<init>(HiveMetaStoreClient.java:239)
        at org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient.<init>(SessionHiveMetaStoreClient.java:70)
        ... 28 more
Caused by: MetaException(message:Version information not found in metastore. )
        at org.apache.hadoop.hive.metastore.ObjectStore.checkSchema(ObjectStore.java:7754)
        at org.apache.hadoop.hive.metastore.ObjectStore.verifySchema(ObjectStore.java:7732)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.hadoop.hive.metastore.RawStoreProxy.invoke(RawStoreProxy.java:101)
        at com.sun.proxy.$Proxy27.verifySchema(Unknown Source)
        at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.getMS(HiveMetaStore.java:583)
        at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.createDefaultDB(HiveMetaStore.java:644)
        at org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.init(HiveMetaStore.java:424)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invokeInternal(RetryingHMSHandler.java:148)
        at org.apache.hadoop.hive.metastore.RetryingHMSHandler.invoke(RetryingHMSHandler.java:107)
        at org.apache.hadoop.hive.metastore.RetryingHMSHandler.<init>(RetryingHMSHandler.java:79)
        ... 32 more

解决方法：

修改hive-site.xml

  <property>
    <name>hive.metastore.schema.verification</name>  
    <value>false</value>  
      <description>  
      Enforce metastore schema version consistency.  
      True: Verify that version information stored in metastore matches with one from Hive jars.  Also disable automatic schema migration attempt. Users are required to manully migrate schema after Hive upgrade which ensures proper metastore schema migration. (Default)  
      False: Warn if the version information stored in metastore doesn't match with one from in Hive jars.  
      </description>
    </property>
  <property>

错误2：The reference to entity “useSSL” must end with the ‘;’ delimiter

详细信息：

[root@hadoop-master bin]# hive
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/hadoop/hive-2.2.0/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/hadoop/hadoop-2.7.4/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
[Fatal Error] hive-site.xml:52:85: The reference to entity "useSSL" must end with the ';' delimiter.
Exception in thread "main" java.lang.RuntimeException: org.xml.sax.SAXParseException; systemId: file:/home/hadoop/hive-2.2.0/conf/hive-site.xml; lineNumber: 52; columnNumber: 85; The reference to entity "useSSL" must end with the ';' delimiter.
        at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2647)
        at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2504)
        at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:2407)
        at org.apache.hadoop.conf.Configuration.get(Configuration.java:1232)
        at org.apache.hadoop.hive.conf.HiveConf.getVar(HiveConf.java:3803)
        at org.apache.hadoop.hive.conf.HiveConf.getVar(HiveConf.java:3872)
        at org.apache.hadoop.hive.conf.HiveConf.initialize(HiveConf.java:3959)
        at org.apache.hadoop.hive.conf.HiveConf.<init>(HiveConf.java:3902)
        at org.apache.hadoop.hive.common.LogUtils.initHiveLog4jCommon(LogUtils.java:82)
        at org.apache.hadoop.hive.common.LogUtils.initHiveLog4j(LogUtils.java:66)
        at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:702)
        at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:686)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: org.xml.sax.SAXParseException; systemId: file:/home/hadoop/hive-2.2.0/conf/hive-site.xml; lineNumber: 52; columnNumber: 85; The reference to entity "useSSL" must end with the ';' delimiter.
        at org.apache.xerces.parsers.DOMParser.parse(Unknown Source)
        at org.apache.xerces.jaxp.DocumentBuilderImpl.parse(Unknown Source)
        at javax.xml.parsers.DocumentBuilder.parse(DocumentBuilder.java:150)
        at org.apache.hadoop.conf.Configuration.parse(Configuration.java:2482)
        at org.apache.hadoop.conf.Configuration.parse(Configuration.java:2470)
        at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:2538)
        ... 17 more

解决方法：

在xml文件中 &符号需要转义 & -> & 于是便解决了

  <property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:mysql://hadoop-master:3306/hive?createDatabaseIfNotExist=true&amp;useSSL=true</value>
    <description>
      JDBC connect string for a JDBC metastore.
      To use SSL to encrypt/authenticate the connection, provide database-specific SSL flag in the connection URL.
      For example, jdbc:postgresql://myhost/db?ssl=true for postgres database.
    </description>
  </property>

错误3： Couldn’t create directory /home/hadoop/hive-2.2.0/tmp/resources

详细信息：

SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/hadoop/hive-2.2.0/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/hadoop/hadoop-2.7.4/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Exception in thread "main" java.lang.RuntimeException: Couldn't create directory /home/hadoop/hive-2.2.0/tmp/resources
        at org.apache.hadoop.hive.ql.util.ResourceDownloader.ensureDirectory(ResourceDownloader.java:126)
        at org.apache.hadoop.hive.ql.util.ResourceDownloader.<init>(ResourceDownloader.java:48)
        at org.apache.hadoop.hive.ql.session.SessionState.<init>(SessionState.java:397)
        at org.apache.hadoop.hive.ql.session.SessionState.<init>(SessionState.java:370)
        at org.apache.hadoop.hive.cli.CliSessionState.<init>(CliSessionState.java:60)
        at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:708)
        at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:686)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

解决方法：

问题的根本原因是用户权限问题。

将hive.downloaded.resources.dir修改为绝对路径：

  <property>
    <name>hive.downloaded.resources.dir</name>
    <value>/home/hadoop/hive-2.2.0/tmp/${hive.session.id}_resources</value>
    <description>Temporary local directory for added resources in the remote file system.</description>
  </property>

如果还有问题，查看登录用户是否有修改该目录文件的权限

错误4：Schema initialization FAILED! Metastore state would be inconsistent !!

[root@hadoopmaster conf]# schematool -initSchema -dbType mysql
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/local/hadoop/hive-2.2.0/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/hadoop/hadoop-2.7.6/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL:        jdbc:derby:;databaseName=metastore_db;create=true
Metastore Connection Driver :    org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User:       APP
Starting metastore schema initialization to 2.1.0
Initialization script hive-schema-2.1.0.mysql.sql
Error: Syntax error: Encountered "<EOF>" at line 1, column 64. (state=42X01,code=30000)
org.apache.hadoop.hive.metastore.HiveMetaException: Schema initialization FAILED! Metastore state would be inconsistent !!
Underlying cause: java.io.IOException : Schema script failed, errorcode 2
Use --verbose for detailed stacktrace.
*** schemaTool failed ***

解决方法：

方法1：首先查看Metastore connection URL、Metastore Connection Driver和Metastore connection User三个选项的内容与hive-site.xml信息里面的内容一样，如果不正确请修改。

方法2：删除$HIVE_HOME/conf目录里面的metastore_db，并删除

然后重新格式化

错误5：The root scratch dir: /home/hadoop/hive-2.2.0/hive on HDFS should be writable. Current permissions are: rwxr-xr-x

[root@hadoop-master bin]# ./hive
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/hadoop/hive-2.2.0/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/hadoop/hadoop-2.7.4/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]

Logging initialized using configuration in jar:file:/home/hadoop/hive-2.2.0/lib/hive-common-2.2.0.jar!/hive-log4j2.properties Async: true
Exception in thread "main" java.lang.RuntimeException: The root scratch dir: /home/hadoop/hive-2.2.0/hive on HDFS should be writable. Current permissions are: rwxr-xr-x
        at org.apache.hadoop.hive.ql.session.SessionState.createRootHDFSDir(SessionState.java:728)
        at org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:658)
        at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:586)
        at org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:549)
        at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:750)
        at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:686)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

错误分析：

从The root scratch dir: /home/hadoop/hive-2.2.0/hive on HDFS should be writable. Current permissions are: rwxr-xr-x可以看出。系统要求在hdfs上用户应该具备写权限，而从报错可以看出只有所有者具有写权限，所有组合其他用户不具备。

解决方法：

使用hadoop fs hadoop fs -chmod -R dir修改目录权限，如下

[root@hadoop-master bin]# hadoop fs -chmod 777 /home/hadoop/hive-2.2.0/hive