hive安装-构建在hadoop上的数据仓库

1.安装

*  解压官网下载的安装包     tar -xvf apache-hive-2.1.0.tar.gz
* vim /etc/profile
* 文件结尾处添加
    * export $HIVE_HOME=/usr/lib/apache-hive-2.1.0
    * export $PATH=$PATH:$HIVE_HOME/bin

* source /etc/profile使配置文件生效

2.运行hive看是否安装成功。错误如下

which: no hbase in
(/usr/lib/apache-hive-2.1.0-bin/bin:/usr/lib/hadoop-2.7.3/bin:/usr/lib/hadoop-2.7.3/sbin:/usr/lib/jdk1.8.0_112/bin:/usr/lib/jdk1.8.0_112/jre/bin:/usr/lib/hadoop-2.7.3/bin:/usr/lib/hadoop-2.7.3/sbin:/usr/lib/jdk1.8.0_112/bin:/usr/lib/jdk1.8.0_112/jre/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/bin:/sbin:/root/bin)
SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found
binding in
[jar:file:/usr/lib/apache-hive-2.1.0-bin/lib/log4j-slf4j-impl-2.4.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/usr/lib/hadoop-2.7.3/share/hadoop/common/lib/slf4j-log4j12-1.7.10.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
explanation. SLF4J: Actual binding is of type
[org.apache.logging.slf4j.Log4jLoggerFactory]

Logging initialized using configuration in
jar:file:/usr/lib/apache-hive-2.1.0-bin/lib/hive-common-2.1.0.jar!/hive-log4j2.properties
Async: true Exception in thread “main” java.lang.RuntimeException:
org.apache.hadoop.hive.ql.metadata.HiveException:
org.apache.hadoop.hive.ql.metadata.HiveException:
MetaException(message:Hive metastore database is not initialized.
Please use schematool (e.g. ./schematool -initSchema -dbType …) to
create the schema. If needed, don’t forget to include the option to
auto-create the underlying database in your JDBC connection string
(e.g. ?createDatabaseIfNotExist=true for mysql))
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:578)
at org.apache.hadoop.hive.ql.session.SessionState.beginStart(SessionState.java:518)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:705)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:641)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
at org.apache.hadoop.util.RunJar.main(RunJar.java:136) Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
org.apache.hadoop.hive.ql.metadata.HiveException:
MetaException(message:Hive metastore database is not initialized.
Please use schematool (e.g. ./schematool -initSchema -dbType …) to
create the schema. If needed, don’t forget to include the option to
auto-create the underlying database in your JDBC connection string
(e.g. ?createDatabaseIfNotExist=true for mysql))
at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:226)
at org.apache.hadoop.hive.ql.metadata.Hive.(Hive.java:366)
at org.apache.hadoop.hive.ql.metadata.Hive.create(Hive.java:310)
at org.apache.hadoop.hive.ql.metadata.Hive.getInternal(Hive.java:290)
at org.apache.hadoop.hive.ql.metadata.Hive.get(Hive.java:266)
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:545)
… 9 more Caused by: org.apache.hadoop.hive.ql.metadata.HiveException:
MetaException(message:Hive metastore database is not initialized.
Please use schematool (e.g. ./schematool -initSchema -dbType …) to
create the schema. If needed, don’t forget to include the option to
auto-create the underlying database in your JDBC connection string
(e.g. ?createDatabaseIfNotExist=true for mysql))
at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3593)
at org.apache.hadoop.hive.ql.metadata.Hive.reloadFunctions(Hive.java:236)
at org.apache.hadoop.hive.ql.metadata.Hive.registerAllFunctionsOnce(Hive.java:221)
… 14 more Caused by: MetaException(message:Hive metastore database is not initialized. Please use schematool (e.g. ./schematool
-initSchema -dbType …) to create the schema. If needed, don’t forget to include the option to auto-create the underlying database in your
JDBC connection string (e.g. ?createDatabaseIfNotExist=true for
mysql))
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3364)
at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3336)
at org.apache.hadoop.hive.ql.metadata.Hive.getAllFunctions(Hive.java:3590)
… 16 more

解决办法：

* cd $HIVE_HOME/bin
* 初始化schema      ./schematool -initSchema -dbType derby

3.运行hive检查是否成功。错误如下：

Exception in thread “main”Java.lang.RuntimeException:
java.lang.IllegalArgumentException:java.net.URISyntaxException:
Relative path in absolute
URI:${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D
at org.apache.Hadoop.hive.ql.session.SessionState.start(SessionState.java:444)
at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:672)
at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:616)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at org.apache.hadoop.util.RunJar.main(RunJar.java:160) Caused by: java.lang.IllegalArgumentException:java.net.URISyntaxException:
Relative path in absolute
URI:${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D
at org.apache.hadoop.fs.Path.initialize(Path.java:148)
at org.apache.hadoop.fs.Path.(Path.java:126)
at org.apache.hadoop.hive.ql.session.SessionState.createSessionDirs(SessionState.java:487)
at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:430)
… 7more Caused by: java.net.URISyntaxException:Relative path in absolute URI:${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D
at java.net.URI.checkPath(URI.java:1804)
at java.net.URI.(URI.java:752)
at org.apache.hadoop.fs.Path.initialize(Path.java:145)
… 10more

解决办法：

* 在$HIVE_HOME/conf创建配置文件 cp hive-default.xml.template hive-site.xml
* 将四处 system.io.tmpdir全部替换为自己制定的特定目录，比如/usr/lib/apache-hive-2.1.0/tmp

4.错误如下

which: no hbase in
(/usr/lib64/qt-3.3/bin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/java/jdk1.8.0/bin:/root/bin:/usr/apache-hive-2.1.0-bin/bin:/usr/java/jdk1.8.0/bin:/usr/apache-hive-2.1.0-bin/bin:/usr/java/jdk1.8.0/bin:/usr/hadoop/hadoop-2.6.4/bin:/usr/apache-hive-2.1.0-bin/bin) SLF4J: Class path contains multiple SLF4J bindings. SLF4J: Found
binding in
[jar:file:/usr/apache-hive-2.1.0-bin/lib/log4j-slf4j-impl-2.4.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in
[jar:file:/usr/hadoop/hadoop-2.6.4/share/hadoop/common/lib/slf4j-log4j12-1.7.5.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an
explanation. SLF4J: Actual binding is of type
[org.apache.logging.slf4j.Log4jLoggerFactory] Metastore connection
URL: jdbc:derby:;databaseName=metastore_db;create=true Metastore
Connection Driver : org.apache.derby.jdbc.EmbeddedDriver Metastore
connection User: APP Starting metastore schema initialization to
2.1.0Initialization script hive-schema-2.1.0.derby.sql Error: FUNCTION ‘NUCLEUS_ASCII’ already exists. (state=X0Y68,code=30000)
org.apache.hadoop.hive.metastore.HiveMetaException: Schema
initialization FAILED! Metastore state would be inconsistent !!
Underlying cause: java.io.IOException : Schema script failed,
errorcode 2 Use –verbose for detailed stacktrace.
* schemaTool failed *

解决办法如下：

* cd $HIVE_HOME/scripts/metastore/upgrade/derby 
* 修改错误堆栈中的脚本hive-schema-2.1.0.derby.sql，注释上面两句。注释方式为在句首添加--