hive数据库初始化问题

最新推荐文章于 2024-05-24 11:27:43 发布

血小板platelet

最新推荐文章于 2024-05-24 11:27:43 发布

阅读量2.9k

点赞数 4

分类专栏：大数据文章标签： hive hadoop 数据库 big data apache

本文链接：https://blog.csdn.net/plt524/article/details/122575830

版权

大数据专栏收录该内容

1 篇文章 0 订阅

订阅专栏

版本信息

hadoop 3.2.2
hbase 1.7.1
mysql 5.7.35
hive 2.3.9

报错信息

java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkArgument

[root@centos76-1 conf]# schematool -dbType mysql -initSchema
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Exception in thread "main" java.lang.NoSuchMethodError: com.google.common.base.Preconditions.checkArgument(ZLjava/lang/String;Ljava/lang/Object;)V
        at org.apache.hadoop.conf.Configuration.set(Configuration.java:1357)
        at org.apache.hadoop.conf.Configuration.set(Configuration.java:1338)
        at org.apache.hadoop.mapred.JobConf.setJar(JobConf.java:536)
        at org.apache.hadoop.mapred.JobConf.setJarByClass(JobConf.java:554)
        at org.apache.hadoop.mapred.JobConf.<init>(JobConf.java:448)
        at org.apache.hadoop.hive.conf.HiveConf.initialize(HiveConf.java:4051)
        at org.apache.hadoop.hive.conf.HiveConf.<init>(HiveConf.java:4014)
        at org.apache.hive.beeline.HiveSchemaTool.<init>(HiveSchemaTool.java:82)
        at org.apache.hive.beeline.HiveSchemaTool.main(HiveSchemaTool.java:1117)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:323)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:236)

错误原因：
系统找不到这个类所在的jar包或者jar包的版本不一样系统不知道使用哪个。hive启动报错的原因是后者。

解决方法：
com.google.common.base.Preconditions.checkArgument这个类所在的jar包为：guava.jar，在hadoop和hive中版本不一致导致。
删除hive中低版本jar包，将hadoop中高版本的复制到hive的lib中。

[root@centos76-1 lib]# ls $HADOOP_HOME/share/hadoop/common/lib/guava*
/usr/local/hadoop/share/hadoop/common/lib/guava-27.0-jre.jar
[root@centos76-1 lib]# ls $HIVE_HOME/lib/guava*
/usr/local/hive/lib/guava-14.0.1.jar
[root@centos76-1 lib]# rm -f $HIVE_HOME/lib/guava*
[root@centos76-1 lib]# cp /usr/local/hadoop/share/hadoop/common/lib/guava-27.0-jre.jar $HIVE_HOME/lib/

org.apache.hadoop.hive.metastore.HiveMetaException: Failed to get schema version.
Underlying cause: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException : Communications link failure

[root@centos76-1 lib]# schematool -dbType mysql -initSchema       
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL:        jdbc:mysql://centos76-1:3306/hive?createDatabaseIfNotExist=true
Metastore Connection Driver :    com.mysql.jdbc.Driver
Metastore connection User:       root
Fri Dec 24 12:36:45 CST 2021 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 
and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not 
L the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and 
truststore for server certificate verification.
org.apache.hadoop.hive.metastore.HiveMetaException: Failed to get schema version.
Underlying cause: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException : Communications link failure
The last packet successfully received from the server was 1,050 milliseconds ago.  The last packet sent successfully to the server was 1,021 
onds ago.
SQL Error code: 0
Use --verbose for detailed stacktrace.
*** schemaTool failed ***

详细查看报错信息发现有SSL报错:javax.net.ssl.SSLHandshakeException: No appropriate protocol

[root@centos76-1 conf]# schematool -dbType mysql -initSchema --verbose
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL:        jdbc:mysql://192.168.220.140:3306/hive?createDatabaseIfNotExist=true
Metastore Connection Driver :    com.mysql.jdbc.Driver
Metastore connection User:       hive
Sat Dec 25 05:31:18 CST 2021 WARN: Establishing SSL connection without server's identity verification is not recommended. According to MySQL 5.5.45+, 5.6.26+ and 5.7.6+ requirements SSL connection must be established by default if explicit option isn't set. For compliance with existing applications not using SSL the verifyServerCertificate property is set to 'false'. You need either to explicitly disable SSL by setting useSSL=false, or set useSSL=true and provide truststore for server certificate verification.
org.apache.hadoop.hive.metastore.HiveMetaException: Failed to get schema version.
Underlying cause: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException : Communications link failure

The last packet successfully received from the server was 1,313 milliseconds ago.  The last packet sent successfully to the server was 1,251 milliseconds ago.
SQL Error code: 0
org.apache.hadoop.hive.metastore.HiveMetaException: Failed to get schema version.
        at org.apache.hive.beeline.HiveSchemaHelper.getConnectionToMetastore(HiveSchemaHelper.java:77)
        at org.apache.hive.beeline.HiveSchemaTool.getConnectionToMetastore(HiveSchemaTool.java:144)
        at org.apache.hive.beeline.HiveSchemaTool.testConnectionToMetastore(HiveSchemaTool.java:473)
        at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:577)
        at org.apache.hive.beeline.HiveSchemaTool.doInit(HiveSchemaTool.java:563)
        at org.apache.hive.beeline.HiveSchemaTool.main(HiveSchemaTool.java:1145)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:323)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:236)
Caused by: com.mysql.jdbc.exceptions.jdbc4.CommunicationsException: Communications link failure

The last packet successfully received from the server was 1,313 milliseconds ago.  The last packet sent successfully to the server was 1,251 milliseconds ago.
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)
        at com.mysql.jdbc.SQLError.createCommunicationsException(SQLError.java:990)
        at com.mysql.jdbc.ExportControlled.transformSocketToSSLSocket(ExportControlled.java:203)
        at com.mysql.jdbc.MysqlIO.negotiateSSLConnection(MysqlIO.java:4901)
        at com.mysql.jdbc.MysqlIO.proceedHandshakeWithPluggableAuthentication(MysqlIO.java:1659)
        at com.mysql.jdbc.MysqlIO.doHandshake(MysqlIO.java:1226)
        at com.mysql.jdbc.ConnectionImpl.coreConnect(ConnectionImpl.java:2188)
        at com.mysql.jdbc.ConnectionImpl.connectOneTryOnly(ConnectionImpl.java:2219)
        at com.mysql.jdbc.ConnectionImpl.createNewIO(ConnectionImpl.java:2014)
        at com.mysql.jdbc.ConnectionImpl.<init>(ConnectionImpl.java:776)
        at com.mysql.jdbc.JDBC4Connection.<init>(JDBC4Connection.java:47)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
        at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
        at com.mysql.jdbc.Util.handleNewInstance(Util.java:425)
        at com.mysql.jdbc.ConnectionImpl.getInstance(ConnectionImpl.java:386)
        at com.mysql.jdbc.NonRegisteringDriver.connect(NonRegisteringDriver.java:330)
        at java.sql.DriverManager.getConnection(DriverManager.java:664)
        at java.sql.DriverManager.getConnection(DriverManager.java:247)
        at org.apache.hive.beeline.HiveSchemaHelper.getConnectionToMetastore(HiveSchemaHelper.java:73)
        ... 11 more
Caused by: javax.net.ssl.SSLHandshakeException: No appropriate protocol (protocol is disabled or cipher suites are inappropriate)
        at sun.security.ssl.HandshakeContext.<init>(HandshakeContext.java:171)
        at sun.security.ssl.ClientHandshakeContext.<init>(ClientHandshakeContext.java:106)
        at sun.security.ssl.TransportContext.kickstart(TransportContext.java:238)
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:394)
        at sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:373)
        at com.mysql.jdbc.ExportControlled.transformSocketToSSLSocket(ExportControlled.java:188)
        ... 29 more
*** schemaTool failed ***

解决方法：
在hive-site.xml中把ConnectionURL修改为如下所示

  <property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:mysql://192.168.220.140:3306/hive?createDatabaseIfNotExist=true&amp;useSSL=false</value>
  </property>

com.ctc.wstx.exc.WstxParsingException: String ‘–’ not allowed in comment (missing ‘>’?)

[root@centos76-1 lib]# schematool -dbType mysql -initSchema       
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Exception in thread "main" java.lang.RuntimeException: com.ctc.wstx.exc.WstxParsingException: String '--' not allowed in comment (missing '>'?)
 at [row,col,system-id]: [18,7,"file:/usr/local/hive/conf/hive-site.xml"]
        at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:3040)
        at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:2989)
        at org.apache.hadoop.conf.Configuration.loadProps(Configuration.java:2864)
        at org.apache.hadoop.conf.Configuration.addResourceObject(Configuration.java:1012)
        at org.apache.hadoop.conf.Configuration.addResource(Configuration.java:917)
        at org.apache.hadoop.hive.conf.HiveConf.initialize(HiveConf.java:4061)
        at org.apache.hadoop.hive.conf.HiveConf.<init>(HiveConf.java:4014)
        at org.apache.hive.beeline.HiveSchemaTool.<init>(HiveSchemaTool.java:82)
        at org.apache.hive.beeline.HiveSchemaTool.main(HiveSchemaTool.java:1117)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:323)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:236)
Caused by: com.ctc.wstx.exc.WstxParsingException: String '--' not allowed in comment (missing '>'?)
 at [row,col,system-id]: [18,7,"file:/usr/local/hive/conf/hive-site.xml"]
        at com.ctc.wstx.sr.StreamScanner.constructWfcException(StreamScanner.java:621)
        at com.ctc.wstx.sr.StreamScanner.throwParseError(StreamScanner.java:491)
        at com.ctc.wstx.sr.StreamScanner.throwParseError(StreamScanner.java:475)
        at com.ctc.wstx.sr.BasicStreamReader.skipCommentOrCData(BasicStreamReader.java:3592)
        at com.ctc.wstx.sr.BasicStreamReader.skipToken(BasicStreamReader.java:3443)
        at com.ctc.wstx.sr.BasicStreamReader.nextFromProlog(BasicStreamReader.java:2074)
        at com.ctc.wstx.sr.BasicStreamReader.next(BasicStreamReader.java:1181)
        at org.apache.hadoop.conf.Configuration$Parser.parseNext(Configuration.java:3336)
        at org.apache.hadoop.conf.Configuration$Parser.parse(Configuration.java:3130)
        at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:3023)
        ... 14 more

错误原因：
在hive-default.xml.template模版里前面有一段注释的结尾–>，使用它作hive-site.xml时误删了–>
解决方法：
补回来

[root@centos76-1 conf]# head -17 hive-default.xml.template 
<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?><!--
   Licensed to the Apache Software Foundation (ASF) under one or more
   contributor license agreements.  See the NOTICE file distributed with
   this work for additional information regarding copyright ownership.
   The ASF licenses this file to You under the Apache License, Version 2.0
   (the "License"); you may not use this file except in compliance with
   the License.  You may obtain a copy of the License at

       http://www.apache.org/licenses/LICENSE-2.0

   Unless required by applicable law or agreed to in writing, software
   distributed under the License is distributed on an "AS IS" BASIS,
   WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
   See the License for the specific language governing permissions and
   limitations under the License.
--><configuration>

org.apache.hadoop.hive.metastore.HiveMetaException: Failed to load driver

[root@centos76-1 conf]# schematool -dbType mysql -initSchema
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/usr/local/hive/lib/log4j-slf4j-impl-2.6.2.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/usr/local/hadoop/share/hadoop/common/lib/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL:        jdbc:mysql://centos76-1:3306/hive?createDatabaseIfNotExist=true
Metastore Connection Driver :    com.mysql.jdbc.Driver
Metastore connection User:       root
org.apache.hadoop.hive.metastore.HiveMetaException: Failed to load driver
Underlying cause: java.lang.ClassNotFoundException : com.mysql.jdbc.Driver
Use --verbose for detailed stacktrace.
*** schemaTool failed ***

错误原因：jdbc driver下错了。
解决方法：
应该下载下面图里的zip包，从里面解压出.jar文件上传到 $HIVE_HOME/lib/ 下 mysql jdbc driver download