一、安装Hadoop
Hive运行在Hadoop环境之上,因此需要hadoop环境,本次在安装在hadoop完全分布式模式的namennode节点上
请参考:hadoop搭建
二、安装Hive
- 下载
[hadoop@hadoop01 /home/hadoop]$cd /app/
[hadoop@hadoop01 /app]$wget http://archive.apache.org/dist/hive/stable-2/apache-hive-2.3.4-bin.tar.gz
- 解压安装
[hadoop@hadoop01 /app]$cd /app
[hadoop@hadoop01 /app]$ln -s apache-hive-2.3.4-bin hive
[hadoop@hadoop01 /app]$tar zxvf apache-hive-2.3.4-bin.tar.gz
- 配置hadoop用户的hive环境变量
echo -e '##################HIVE环境变量配置#############\nexport HIVE_HOME=/app/hive\nexport PATH=$HIVE_HOME/bin:$PATH' >> ~/.bash_profile&& source ~/.bash_profile&&tail -3 ~/.bash_profile
三、配置hive
3.1 拷贝出hive-site.xml和hive-env.sh配置文件
[hadoop@hadoop01 /app/hive/conf]$cd /app/hive/conf/
[hadoop@hadoop01 /app/hive/conf]$cp hive-default.xml.template hive-site.xml
[hadoop@hadoop01 /app]$cp hive-env.sh.template hive-env.sh
3.2 修改hive-site.xm中系统无法识别的变量${system:java.io.tmpdir}
和${system:user.name}
- 测试
[hadoop@hadoop01 /app/hive/conf]$sed -n 's#${system:java.io.tmpdir}#/app/hive.java.io.tmpdir#pg' hive-site.xml
<value>/app/hive.java.io.tmpdir/${system:user.name}</value>
<value>/app/hive.java.io.tmpdir/${hive.session.id}_resources</value>
<value>/app/hive.java.io.tmpdir/${system:user.name}</value>
<value>/app/hive.java.io.tmpdir/${system:user.name}/operation_logs</value>
[hadoop@hadoop01 /app/hive/conf]$sed -n 's#${system:user.name}#hadoop#pg' hive-site.xml
<value>/app/hive.java.io.tmpdir/hadoop</value>
<value>/app/hive.java.io.tmpdir/hadoop</value>
<value>/app/hive.java.io.tmpdir/hadoop/operation_logs</value>
- 替换
[hadoop@hadoop01 /app/hive/conf]$sed -i 's#${system:java.io.tmpdir}#/app/hive.java.io.tmpdir#g' hive-site.xml
[hadoop@hadoop01 /app/hive/conf]$sed -i 's#${system:user.name}#hadoop#g' hive-site.xml
- 检查
[hadoop@s101 /app/hive/conf]$grep 'hive.java.io.tmpdir' hive-site.xml
<value>/app/hive.java.io.tmpdir/${system:user.name}</value>
<value>/app/hive.java.io.tmpdir/${hive.session.id}_resources</value>
<value>/app/hive.java.io.tmpdir/${system:user.name}</value>
<value>/app/hive.java.io.tmpdir/${system:user.name}/operation_logs</value>
[hadoop@s101 /app/hive/conf]$grep 'hive.java.io.tmpdir/hadoop' hive-site.xml
<value>/app/hive.java.io.tmpdir/hadoop</value>
<value>/app/hive.java.io.tmpdir/hadoop</value>
<value>/app/hive.java.io.tmpdir/hadoop/operation_logs</value>
- 创建出文件夹
[hadoop@hadoop01 /app]$mkdir -p /app/hive.java.io.tmpdir/hadoop
3.3 初始化默认derby数据库
[hadoop@hadoop01 /home/hadoop]$schematool -initSchema -dbType derby
.......
Metastore connection URL: jdbc:derby:;databaseName=metastore_db;create=true
Metastore Connection Driver : org.apache.derby.jdbc.EmbeddedDriver
Metastore connection User: APP
Starting metastore schema initialization to 2.3.0
Initialization script hive-schema-2.3.0.derby.sql
Initialization script completed
schemaTool completed
- 初始化完成后,会在当前目录产生一个metastore_db文件夹,这就是数据库的名称。
HDFS文件系统中也会产生相应的目录
[hadoop@hadoop01 /home/hadoop]$ll /home/hadoop/metastore_db/
[hadoop@hadoop01 /home/hadoop]$hdfs dfs -ls /tmp
Found 2 items
drwx-wx-wx - hadoop supergroup 0 2018-11-26 19:12 /tmp/hive
四、进入hive命令行
- 检查hadoop集群是否启动,如果没有启动需要启动起来
[hadoop@hadoop01 /home/hadoop]start-dfs.sh
[hadoop@hadoop01 /home/hadoop]start-yarn.sh
- 进入hive命令行
[hadoop@hadoop01 /home/hadoop]$hive
...
hive>
五、配置MySQL数据库存储元数据
5.1 修改URL、drivername、username、password等配置信息
[hadoop@hadoop01 /home/hadoop]$vim /app/hive/conf/hive-site.xml
<property>
<name>javax.jdo.option.ConnectionURL</name> <value>jdbc:mysql://hadoop01:3306/hive</value> </property> <property> <name>javax.jdo.option.ConnectionDriverName</name> <value>com.mysql.jdbc.Driver</value> </property> <property> <name>javax.jdo.option.ConnectionUserName</name> <value>hive</value> </property> <property> <name>javax.jdo.option.ConnectionPassword</name> <value>123456</value> </property>
5.2将java的msyql驱动拷贝至apache-hive-2.3.4-bin/lib中
5.3初始化mysql数据
schematool -initSchema -dbType mysql