1.说明
(1)安装Hive之前必须配置好hadoop环境;
(2)安装Hive之前必须配置好JDK;
(3)本文建议安装mysql数据库·;
(4)Hive可以只安装在namenode所在的机器上,如果NameNode有多个,则每个都需要安装,可以不在datanode上安装。
(5)本文hadoop的运行采用伪分布式;
(6)本文中的所有目录(路径)均为本人自定义,访客可根据自己的需求进行更改。
2.版本说明
(1)JDK版本:JDK1.8.0
(2)hadoop版本:Hadoop2.8.3
(3)hive版本:Hive2.3.3
(4)mysql版本:MySql5.7.2
(5)Linux版本:CentOS6.6
3.步骤
(1)下载hive
下载链接为:http://mirrors.hust.edu.cn/apache/hive/
(2)解压hive
将下载好的hive解压到/usr/env/hive/路径下:
tar -zxvf apache-hive-2.1.1-bin.tar.gz -C /usr/env/hive/
(3)配置hive环境变量
进入/etc/profile:
vim /etc/profile
配置hive的安装路径和conf路径:
export HIVE_HOME=/usr/env/hive/hive-2.3.3
export HIVE_CONF_DIR=$HIVE_HOME/conf
export PATH=$PATH:$JAVA_HOME/bin:$HADOOP_HOME/bin:$HADOOP_HOME/sbin:$HIVE_HOME/bin
使/etc/profile生效:
source /etc/profile
(4)创建数据仓库目录
在HDFS上创建数据仓库目录:
hadoop fs -mkdir -p /home/hive/warehouse
设置目录权限:
hadoop fs -chmod -R 777 /home/hive/warehouse
(5)创建临时目录
在HDFS创建Hive的临时目录:
hadoop fs -mkdir -p /tmp/hive
设置目录权限:
hadoop fs -chmod 755-R /tmp/hive
(6)设置配置文件hive-site.xml
-
找到hive-site.xml.template文件
配置文件目录为:
/usr/env/hive/hive-2.3.3/conf
将hive-site.xml.template文件改名为hive-site.xml:
mv hive-site.xml.template hive-site.xml
- 配置数据仓库的目录hive.metastore.warehouse.dir
<property>
<name>hive.metastore.warehouse.dir</name>
<value>/home/hive/warehouse</value>
<description>location of default database for the warehouse</description>
</property>
- 配置hive.exec.scratchdir
hive用来存储不同阶段的map/reduce的执行计划的目录,同时也存储中间输出结果
<property>
<name>hive.exec.scratchdir</name>
<value>/tmp/hive</value>
<description>
HDFS root scratch dir for Hive jobs which gets created with write all (733) permission. For each connecting user, an HDFS scratch dir: ${hive.exec.scratchdir}/<username> is created, with ${hive.scratch.dir.permission}.
</description>
</property>
- 配置hive.exec.local.scratchdir
当hive运行在本地模式时配置
<property>
<name>hive.exec.local.scratchdir</name>
<value>/tmp/hive/root</value>
<description>Local scratch space for Hive jobs</description>
</property>
- 配置hive.scratch.dir.permission
允许在根scratch录中创建的用户特定的scratch目录的权限
<property>
<name>hive.scratch.dir.permission</name>
<value>777</value>
<description>The permission for the user specific scratch directories that get created.</description>
</property>
- 配置hive.downloaded.resources.dir
远程资源下载的临时目录(创建一个tmp目录,本文将该目录创建在解压目录下)
<property>
<name>hive.downloaded.resources.dir</name>
<value>/usr/env/hive/hive-2.3.3/tmp/${hive.session.id}_resources</value>
<description>Temporary local directory for added resources in the remote file system.</description>
</property>
- 将配置文件中${system:user.name}都替换为root
- 配置数据库连接驱动javax.jdo.option.ConnectionDriverName(本文使用的mysql数据库)
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
<description>Driver class name for a JDBC metastore</description>
</property>
- 配置连接数据库的URL javax.jdo.option.ConnectionURL
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://XXX.XXX.XXX.XXX:3306/hive?createDatabaseIfNotExist=true</value>
<description>
JDBC connect string for a JDBC metastore.
To use SSL to encrypt/authenticate the connection, provide database-specific SSL flag in the connection URL.
For example, jdbc:postgresql://myhost/db?ssl=true for postgres database.
</description>
</property>
- 配置数据库登录名 javax.jdo.option.ConnectionUserName
<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>root</value>
<description>Username to use against metastore database</description>
</property>
- 配置数据库登录密码 .javax.jdo.option.ConnectionPassword
<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>XXXXX</value>
<description>password to use against metastore database</description>
</property>
- 将mysql的驱动包保存到hive安装目录下lib文件夹下
本文使用的驱动包为mysql-connector-java-5.1.35.jar,访客可根据自己实际情况而定。
(7)配置hive-env.xml
- 将原始的hive-env.xml.template改名为hive-env.xml
mv hive-env.xml.template hive-env.xml
配置HADOOP_HOME,HIVE_CONF_DIR,HIVE_AUX_JARS_PATH
(8)对mysql数据库进行初始化
schematool -initSchema -dbType mysql
(9)启动hive
进入hive安装目录下的bin目录,运行:
./hive