apache-hive-1.2.2安装教程

1、安装前提

1、hadoop集群（伪分布式也可以）
2、mysql数据库

这里假设hadoop集群和mysql数据库都安装好了。

2、配置mysql数据库

1、创建hive元数据存放库

mysql> create database hivemeta character set = latin1;
Query OK, 1 row affected (0.00 sec)

2、创建hive数据库用户

mysql> grant all on hivemeta.* to hiveuser@'%' identified by 'hiveuser';
Query OK, 0 rows affected (0.00 sec)

mysql> grant all on hivemeta.* to hiveuser@'localhost' identified by 'hiveuser';
Query OK, 0 rows affected (0.00 sec)

mysql> grant all on hivemeta.* to hiveuser@'hbaselx' identified by 'hiveuser';
Query OK, 0 rows affected (0.05 sec)

mysql> use mysql;
Database changed
mysql> select user,host,password from user where user='hiveuser';
+----------+-----------+-------------------------------------------+
| user     | host      | password                                  |
+----------+-----------+-------------------------------------------+
| hiveuser | %         | *9B0A19544652EC0C7748682AB93554C85080C326 |
| hiveuser | localhost | *9B0A19544652EC0C7748682AB93554C85080C326 |
+----------+-----------+-------------------------------------------+
2 rows in set (0.00 sec)
mysql> exit
Bye
[hadoop@hbaselx ~]$ mysql -uhiveuser -p
Enter password: 
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 12
Server version: 5.1.71 Source distribution

Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql> 

3、配置hive

1、解压

[hadoop@hbaselx ~]$ ls
apache-hive-1.2.2-bin.tar.gz  app  hadoop-2.6.5.tar.gz  jdk-8u172-linux-x64.tar.gz
[hadoop@hbaselx ~]$ tar -zxvf apache-hive-1.2.2-bin.tar.gz -C app

2、复制配置文件

[hadoop@hbaselx ~]$ cd app
[hadoop@hbaselx app]$ ls
apache-hive-1.2.2-bin  hadoop-2.6.5  jdk1.8.0_172
[hadoop@hbaselx app]$ cd apache-hive-1.2.2-bin/conf
[hadoop@hbaselx conf]$ ls
beeline-log4j.properties.template  hive-env.sh.template                 hive-log4j.properties.template
hive-default.xml.template          hive-exec-log4j.properties.template  ivysettings.xml
[hadoop@hbaselx conf]$ cp hive-default.xml.template hive-site.xml
[hadoop@hbaselx conf]$ cp hive-env.sh.template hive-env.sh
[hadoop@hbaselx conf]$ cp hive-log4j.properties.template hive-log4j.properties
[hadoop@hbaselx conf]$ cp  hive-exec-log4j.properties.template  hive-exec-log4j.properties
[hadoop@hbaselx conf]$ 

3、在hdfs上创建hive需要的目录

[hadoop@hbaselx conf]$ hdfs dfs -mkdir -p /hive/warehouse
18/06/14 10:54:21 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[hadoop@hbaselx conf]$ hdfs dfs -mkdir -p /hive/log
18/06/14 10:54:28 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[hadoop@hbaselx conf]$ hdfs dfs -mkdir -p /hive/tmp
18/06/14 10:54:33 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
[hadoop@hbaselx conf]$ 
[hadoop@hbaselx apache-hive-1.2.2-bin]$ hdfs dfs -chmod -R 777 /hive

4、在本地创建如下目录

[hadoop@hbaselx apache-hive-1.2.2-bin]$ mkdir log
[hadoop@hbaselx apache-hive-1.2.2-bin]$ mkdir exec
[hadoop@hbaselx apache-hive-1.2.2-bin]$ mkdir downloadedsource

5、配置hive-site.xml
hive-site.xml文件里配置项很多，我们只需要修改下面这些即可。

<property>
  <name>javax.jdo.option.ConnectionDriverName</name>
  <value>com.mysql.jdbc.Driver</value>
  <description>Driver class name for a JDBC metastore</description>
</property>
<property>
    <name>javax.jdo.option.ConnectionUserName</name>
    <value>hiveuser</value>
    <description>Username to use against metastore database</description>
</property>

 <property>
    <name>javax.jdo.option.ConnectionPassword</name>
    <value>hiveuser</value>
    <description>password to use against metastore database</description>
  </property>

  <property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>jdbc:mysql://192.168.16.66:3306/hivemeta</value>
    <description>JDBC connect string for a JDBC metastore</description>
  </property>

   <property>
    <name>hive.metastore.uris</name>
    <value/>
    <description>Thrift URI for the remote metastore. Used by metastore client to connect to remote metastore.</description>  
  </property>

  <property>
    <name>hive.metastore.warehouse.dir</name>
    <value>/hive/warehouse</value>   //hive数据文件在hdfs上存放目录
    <description>location of default database for the warehouse</description>
  </property>

    <property>
    <name>hive.exec.local.scratchdir</name>
    <value>/home/hadoop/app/apache-hive-1.2.2-bin/exec</value>
    <description>Local scratch space for Hive jobs</description>
  </property>

    <property>
    <name>hive.downloaded.resources.dir</name>
    <value>/home/hadoop/app/apache-hive-1.2.2-bin/downloadedsource</value>
    <description>Temporary local directory for added resources in the remote file system.</description>
  </property>

6、配置hive-exec-log4j.properties和hive-log4j.properties
修改hive.log.dir为：

hive.log.dir=/home/hadoop/app/apache-hive-1.2.2-bin/log

7、拷贝mysql连接驱动包到${HIVE_HOME}/lib

[hadoop@hbaselx ~]$ cp mysql-connector-java-5.1.38.jar app/apache-hive-1.2.2-bin/lib

4、启动hive metastore服务

[hadoop@hbaselx bin]$ ./hive --service metastore &
[1] 25981
[hadoop@hbaselx bin]$ Starting Hive Metastore Server

[hadoop@hbaselx bin]$ 

5、启动 hive cli

[hadoop@hbaselx bin]$ ./hive

Logging initialized using configuration in file:/home/hadoop/app/apache-hive-1.2.2-bin/conf/hive-log4j.properties
hive> create table t1
    > (id int,
    > name string);
OK
Time taken: 1.665 seconds
hive> show tables;
OK
t1
Time taken: 0.236 seconds, Fetched: 1 row(s)
hive> select * from t1;
OK
Time taken: 0.672 seconds

安装hive遇到的问题

问题1：

[hadoop@hbaselx bin]$ ./hive

Logging initialized using configuration in file:/home/hadoop/app/apache-hive-1.2.2-bin/conf/hive-log4j.properties
[ERROR] Terminal initialization failed; falling back to unsupported
java.lang.IncompatibleClassChangeError: Found class jline.Terminal, but interface was expected
        at jline.TerminalFactory.create(TerminalFactory.java:101)
        at jline.TerminalFactory.get(TerminalFactory.java:158)
        at jline.console.ConsoleReader.<init>(ConsoleReader.java:229)
        at jline.console.ConsoleReader.<init>(ConsoleReader.java:221)
        at jline.console.ConsoleReader.<init>(ConsoleReader.java:209)
        at org.apache.hadoop.hive.cli.CliDriver.setupConsoleReader(CliDriver.java:787)
        at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:721)
        at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681)
        at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

Exception in thread "main" java.lang.IncompatibleClassChangeError: Found class jline.Terminal, but interface was expected
        at jline.console.ConsoleReader.<init>(ConsoleReader.java:230)
        at jline.console.ConsoleReader.<init>(ConsoleReader.java:221)
        at jline.console.ConsoleReader.<init>(ConsoleReader.java:209)
        at org.apache.hadoop.hive.cli.CliDriver.setupConsoleReader(CliDriver.java:787)
        at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:721)
        at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:681)
        at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:606)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

解决方式：
把 ${HADOOP_HOME}/share/hadoop/yarn/lib/的 jline 删了,换成hive自带的2.12版本的

rm ${HADOOP_HOME}/share/hadoop/yarn/lib/jline-0.9.94.jar
cp ${HIVE_HOME}/lib/jline-2.12.jar ${HADOOP_HOME}/share/hadoop/yarn/lib/

问题2：

[hadoop@hbaselx bin]$ ./hive

Logging initialized using configuration in file:/home/hadoop/app/apache-hive-1.2.2-bin/conf/hive-log4j.properties
hive> create table t1(
    > id int,
    > name string
    > );
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:For direct MetaStore DB connections, we don't support retries at the client level.)

解决方式：
数据库的字符集不对，应该是latin1。

mysql> show create database hivemeta\G
*************************** 1. row ***************************
       Database: hivemeta
Create Database: CREATE DATABASE `hivemeta` /*!40100 DEFAULT CHARACTER SET utf8 */
1 row in set (0.00 sec)

mysql> drop database hivemeta;
Query OK, 22 rows affected (0.16 sec)

mysql> create database hivemeta;
Query OK, 1 row affected (0.00 sec)

mysql> show create database hivemeta\G
*************************** 1. row ***************************
       Database: hivemeta
Create Database: CREATE DATABASE `hivemeta` /*!40100 DEFAULT CHARACTER SET latin1 */
1 row in set (0.00 sec)

然后重新启动metastore服务即可。