hive 安装部署

1. hive 安装

hive 下载地址

1、下载解压：

// 这里选择的是 1.2.1 的版本
[hadoop@hadoop1 apps]$ tar -zxvf apache-hive-1.2.1-bin.tar.gz 

// 修改名称为 hive
[hadoop@hadoop1 apps]$ mv apache-hive-1.2.1-bin hive

2、修改环境变量：

[hadoop@hadoop1 apps]$ vim ~/.bashrc

export HIVE_HOME=/home/hadoop/apps/hive
export PATH=$PATH:$HIVE_HOME/bin

[hadoop@hadoop1 apps]$ source ~/.bashrc

3、配置 hive-env.sh：

[hadoop@hadoop1 apps]$ cd hive/conf/
[hadoop@hadoop1 conf]$ cp hive-env.sh.template hive-env.sh
[hadoop@hadoop1 conf]$ vim hive-env.sh

# Set HADOOP_HOME to point to a specific hadoop install directory
# HADOOP_HOME=${bin}/../../hadoop
// 1. 配置 hadoop 地址
export HADOOP_HOME=/home/hadoop/apps/hadoop-2.7.5

// 配置 hive 配置文件
# Hive Configuration Directory can be controlled by:
# export HIVE_CONF_DIR=
export HIVE_CONF_DIR=/home/hadoop/apps/hive/conf

4、启动 hadoop、hdfs，创建 /tmp 和 /user/hive/warehouse两个目录并修改他们的同组权限可写：

// hive 表默认存储在 默认的文件系统中的 /user/hive/warehouse 
[hadoop@hadoop1 conf]$ hadoop fs -mkdir /tmp
[hadoop@hadoop1 conf]$ hadoop fs -chmod g+w /tmp
[hadoop@hadoop1 conf]$ hadoop fs -mkdir -p /user/hive/warehouse
[hadoop@hadoop1 conf]$ hadoop fs -chmod g+w /user/hive/warehouse

2. MySQL 安装

hive 默认采用 derby 数据库，一般不推荐使用，不能多个客户端同时连接，生产环境推荐使用 MySQL

1、卸载系统自带 MySQL：

// 检查
[hadoop@hadoop1 apps]$ rpm -qa|grep mysql
mysql-libs-5.1.73-7.el6.x86_64

// 卸载
rpm -e --nodeps mysql-libs-5.1.73-7.el6.x86_64

2、安装：

// 服务端安装
[hadoop@hadoop1 apps]$ sudo rpm -ivh MySQL-server-5.6.24-1.el6.x86_64.rpm
[sudo] password for hadoop:

// 服务端安装完毕后，可用命令查看产生的随机密码
[hadoop@hadoop1 mysql-libs]$ sudo cat /root/.mysql_secret
# The random password set for the root user at Sun Nov 14 16:59:38 2021 (local time): Y4LsdXetH9s8f6XH

// 客户端安装
[hadoop@hadoop1 apps]$ sudo rpm -ivh MySQL-client-5.6.24-1.el6.x86_64.rpm

3、修改密码：

[root@hadoop1 mysql-libs]# mysql -uroot -pY4LsdXetH9s8f6XH
Warning: Using a password on the command line interface can be insecure.
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 1
Server version: 5.6.24

Copyright (c) 2000, 2015, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

// 修改密码为 hadoop
mysql> SET PASSWORD=PASSWORD('hadoop');
Query OK, 0 rows affected (0.00 sec)

4、修改权限使其能够在任意位置连接 MySQL：

mysql> select user,password,host from mysql.user;
+------+-------------------------------------------+-----------+
| user | password                                  | host      |
+------+-------------------------------------------+-----------+
| root | *B34D36DA2C3ADBCCB80926618B9507F5689964B6 | localhost |
| root | *39492AB8B965E7E29B4E43035AF0C2089049DA6F | hadoop1   |
| root | *39492AB8B965E7E29B4E43035AF0C2089049DA6F | 127.0.0.1 |
| root | *39492AB8B965E7E29B4E43035AF0C2089049DA6F | ::1       |
+------+-------------------------------------------+-----------+
4 rows in set (0.00 sec)

// 删除其他用户，只保留 localhost，只有该 host 密码为 hadoop，其他用户密码未知
mysql> drop user root@'hadoop1';
Query OK, 0 rows affected (0.00 sec)

mysql> drop user root@'127.0.0.1';
Query OK, 0 rows affected (0.00 sec)

mysql> drop user root@'::1';
Query OK, 0 rows affected (0.00 sec)

// 更新 host 为 %
mysql> update mysql.user set host='%' where user='root';
Query OK, 1 row affected (0.00 sec)
Rows matched: 1  Changed: 1  Warnings: 0

mysql> select user,password,host from mysql.user;
+------+-------------------------------------------+------+
| user | password                                  | host |
+------+-------------------------------------------+------+
| root | *B34D36DA2C3ADBCCB80926618B9507F5689964B6 | %    |
+------+-------------------------------------------+------+
1 row in set (0.00 sec)

注意：更新了权限后，需要重启 MySQL，重启命令：service mysql restart

3. hive 元数据配置为 MySQL

3.1 驱动拷贝

将 mysql-connector-java-5.1.27-bin.jar 包拷贝到 hive/lib 中：

[root@hadoop1 mysql-libs]# tar -zxvf mysql-connector-java-5.1.27.tar.gz^C

[root@hadoop1 mysql-libs]# ls
MySQL-client-5.6.24-1.el6.x86_64.rpm  mysql-connector-java-5.1.27  mysql-connector-java-5.1.27.tar.gz  MySQL-server-5.6.24-1.el6.x86_64.rpm

[root@hadoop1 mysql-libs]# cp mysql-connector-java-5.1.27/mysql-connector-java-5.1.27-bin.jar /home/hadoop/apps/hive/lib/

3.2 配置 Metastore 到 MySql

1、hive/conf 创建 hive-site.xml：

[hadoop@hadoop1 conf]$ vim hive-site.xml

<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
        <property>
                <name>javax.jdo.option.ConnectionURL</name>
                <value>jdbc:mysql://hadoop1:3306/hivedb?createDatabaseIfNotExist=true</value>
                <description>JDBC connect string for a JDBC metastore</description>
                <!-- 如果 mysql 和 hive 在同一个服务器节点，那么请更改 hadoop02 为 localhost -->
        </property>
        <property>
                <name>javax.jdo.option.ConnectionDriverName</name>
                <value>com.mysql.jdbc.Driver</value>
                <description>Driver class name for a JDBC metastore</description>
        </property>
        <property>
                <name>javax.jdo.option.ConnectionUserName</name>
                <value>root</value>
                <description>username to use against metastore database</description>
        </property>
        <property>
                <name>javax.jdo.option.ConnectionPassword</name>
                <value>hadoop</value>
        <description>password to use against metastore database</description>
        </property>
</configuration>

注意：启动 hive 时，MySQL 权限需要配置好，否则可能会提示无权限访问

3.3 hive 启动测试

[hadoop@hadoop1 conf]$ hive
Logging initialized using configuration in jar:file:/home/hadoop/apps/hive/lib/hive-common-1.2.1.jar!/hive-log4j.properties
hive> show databases;
OK
default
Time taken: 2.924 seconds, Fetched: 1 row(s)

// 创建数据库 hive_1
hive>  create database hive_1;
OK
Time taken: 0.366 seconds

hive> show databases;
OK
default
hive_1
Time taken: 0.071 seconds, Fetched: 2 row(s)

// 使用数据库
hive> use hive_1;
OK
Time taken: 0.12 seconds

// 查看当前正在使用的数据库
hive> select current_database();
OK
hive_1
Time taken: 3.961 seconds, Fetched: 1 row(s)

// 创建表 student
hive> create table student(id int, name string, sex string, age int, department string) row format delimited fields terminated by ",";
OK
Time taken: 0.457 seconds
hive> show tables;
OK
student
Time taken: 0.111 seconds, Fetched: 1 row(s)

// 查询
hive> select * from student;
OK
Time taken: 0.39 seconds

创建成功可在 web-ui 上查看：http://192.168.131.137:50070/explorer.html#/user/hive/warehouse

参考文章

• Hive安装与配置详解
• Hive学习之路 （二）Hive安装