Hadoop2.5.1安装Hive0.13.1

最新推荐文章于 2023-01-05 17:44:35 发布

猫又全宗

最新推荐文章于 2023-01-05 17:44:35 发布

阅读量679

点赞数

分类专栏： hadoop 文章标签： hadoop hive

本文链接：https://blog.csdn.net/DuSizhong/article/details/41677877

版权

hadoop 专栏收录该内容

7 篇文章 0 订阅

订阅专栏

接上一篇文章继续配置。

机器列表：

127.0.0.1       localhost
192.168.11.99   master
192.168.11.97   master2
192.168.12.174  slave1
192.168.12.178  slave2
192.168.12.18   slave3
192.168.11.94   slave4
192.168.12.179  zookeeper1
192.168.12.182  zookeeper2
192.168.12.185  zookeeper3
192.168.12.11   hive

注意：安装hive的机器上要有与slave相同的hadoop设置（lib和环境变量），可以不添加到hadoop集群的datanode中。

1 下载安装hive

下载apache-hive-0.13.1-bin.tar.gz，解压并重命名为/usr/hive。

编辑/etc/profile，添加环境变量：

export HIVE_HOME=/usr/hive
export PATH=/usr/hadoop/bin:/usr/hadoop/sbin:$HIVE_HOME/bin:$JAVA_HOME/bin:$PATH
export CLASSPATH=.:/usr/hadoop/share/hadoop/common/:/usr/hadoop/share/hadoop/common/lib/:/usr/hive/lib/:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar

2 改用mysql作为元数据存储

官网关于元数据存储的设置：Hive Metastore Administration

hive默认是用derby作为元数据存储的。因为derby数据库使用的局限性，推荐采用mysql作为元数据库。
derby存在的缺陷：
1.derby不能多个客户端登录
2.derby登录必须在相同目录下，否则可能会找不到所创建的表。
比如在/hive目录下启动hive程序，那么所创建的表就会存储在/hive下面保存。如果在/home下面，所创建的表就会在/home下面保存。

首先查看系统是否已经安装了mysql

rpm -qa | grep mysql

有的话可以选择卸载并更新

rpm -e mysql

查看yum源

yum list | grep mysql

安装mysql

yum install -y mysql-server mysql mysql-devel

启动服务

service mysqld start

设置密码

mysqladmin -u root password '123'

[root@hive hive]# mysql -u root -p
Enter password: 
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 34032
Server version: 5.1.73 Source distribution

Copyright (c) 2000, 2013, Oracle and/or its affiliates. All rights reserved.

Oracle is a registered trademark of Oracle Corporation and/or its
affiliates. Other names may be trademarks of their respective
owners.

Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.

mysql>

创建元数据数据库

create database hivemeta

授权

GRANT ALL PRIVILEGES ON *.* TO 'root'@'%' IDENTIFIED BY '123' ;
flush privileges;

3 配置hive

重命名hive-env.sh.template为hive-env.sh并配置如下项目：

# The heap size of the jvm stared by hive shell script can be controlled via:
#
export HADOOP_HEAPSIZE=1024

# Set HADOOP_HOME to point to a specific hadoop install directory
export HADOOP_HOME=/usr/hadoop

# Hive Configuration Directory can be controlled by:
export HIVE_CONF_DIR=/usr/hive/conf

# Folder containing extra ibraries required for hive compilation/execution can be controlled by:
export HIVE_AUX_JARS_PATH=/usr/hive/lib

重命名hive-default.xml.template为hive-site.xml并配置如下项：

<property>
  <name>javax.jdo.option.ConnectionURL</name>
  <value>jdbc:mysql://192.168.12.11:3306/hiveMeta?createDatabaseIfNotExist=true</value>
  <description>JDBC connect string for a JDBC metastore</description>
</property>

<property>
  <name>javax.jdo.option.ConnectionDriverName</name>
  <value>com.mysql.jdbc.Driver</value>
  <description>Driver class name for a JDBC metastore</description>
</property>

<property>
  <name>javax.jdo.option.ConnectionUserName</name>
  <value>root</value>
  <description>username to use against metastore database</description>
</property>

<property>
  <name>javax.jdo.option.ConnectionPassword</name>
  <value>123</value>
  <description>password to use against metastore database</description>
</property>

4 测试

启动hive

[root@hive ~]# hive
14/12/02 13:52:46 WARN conf.HiveConf: DEPRECATED: hive.metastore.ds.retry.* no longer has any effect.  Use hive.hmshandler.retry.* instead

Logging initialized using configuration in jar:file:/usr/hive/lib/hive-common-0.13.1.jar!/hive-log4j.properties
hive> show tables;
OK