Hive学习笔记：01-Hive环境的搭建

环境准备

• Hadoop3.3.1（hadoop集群环境已经搭建好）
• JDK1.8(版本过高可能会出现报错)
• Mysql5.7

安装步骤

1、下载Hive的安装包

Hive安装包的官网地址是https://dlcdn.apache.org/hive/hive-3.1.3/apache-hive-3.1.3-bin.tar.gz

在shell中执行命令，下载hive的安装包

[root@k8s-node3 software]# wget https://dlcdn.apache.org/hive/hive-3.1.3/apache-hive-3.1.3-bin.tar.gz

2、解压文件

[root@k8s-node3 software]# tar -xzvf apache-hive-3.1.3-bin.tar.gz 

3、配置hive的环境变量

touch /etc/profile.d/apache-hive.sh ,在文件中添加下列内容：

export HIVE_HOME=/home/software/apache-hive-3.1.3-bin
export PATH=$HIVE_HOME/bin:$PATH

刷新环境变量，使配置生效

[root@k8s-node3 apache-hive-3.1.3-bin]# view /etc/profile.d/apache-hive.sh 
[root@k8s-node3 apache-hive-3.1.3-bin]# source /etc/profile.d/apache-hive.sh 

4、在HDFS中创建hive的相关目录

由于hive依赖hadoop，所以必须在HIVE的安装机器上已经部署好hadoop环境，并且配置好hadoop的环境变量

[root@k8s-node3 ~]# hdfs dfs -mkdir /tmp
[root@k8s-node3 ~]# hdfs dfs -mkdir -p /user/hive/warehouse 
[root@k8s-node3 ~]# hdfs dfs -chmod g+w /tmp 
[root@k8s-node3 ~]# hdfs dfs -chmod g+w /user/hive/warehouse

5、hive-site.xml配置文件

在/home/software/apache-hive-3.1.3-bin/conf目录中创建hive-site.xml配置文件，配置文件的内容如下:

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
  <!-- jdbc  URL -->
  <property>
    <name>javax.jdo.option.ConnectionURL</name>
      <!--规划mysql中存储元数据的数据库名为hive，当初始化数据时不存在时自动创建-->
    <value>jdbc:mysql://192.168.0.44:3306/hive?createDatabaseIfNotExist=true&amp;useSSL=false</value>
  </property>
  <!-- jdbc  Driver-->
  <property>
    <name>javax.jdo.option.ConnectionDriverName</name>
    <value>com.mysql.jdbc.Driver</value>
  </property>
  <!--数据库用户名-->
  <property>
    <name>javax.jdo.option.ConnectionUserName</name>
    <value>root</value>
  </property>
  <!--数据库密码-->
  <property>
    <name>javax.jdo.option.ConnectionPassword</name>
    <value>mysqlroot</value>
  </property>
 
  <!-- Hive 元数据存储版本的验证 -->
  <property>
	<name>hive.metastore.schema.verification</name>
	<value>false</value>
  </property>
  <!--元数据存储授权-->
 <property>
   <name>hive.metastore.event.db.notification.api.auth</name>
   <value>false</value>
 </property>
<!-- Hive 默认在 HDFS 的工作目录 -->
  <property>
    <name>hive.metastore.warehouse.dir</name>
    <value>/user/hive/warehouse</value>
 </property>
</configuration>

6、上传mysql的驱动包

将mysql-connector-java-5.1.32-bin.jar上传到/home/software/apache-hive-3.1.3-bin/lib目录中

7、初始化元数据存储

[root@k8s-node3 conf]# schematool -initSchema -dbType mysql
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/software/apache-hive-3.1.3-bin/lib/log4j-slf4j-impl-2.17.1.jar!/org/s]
SLF4J: Found binding in [jar:file:/home/software/hadoop-3.3.1/share/hadoop/common/lib/slf4j-log4j12-1.7.30.jaer.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Metastore connection URL:	 jdbc:mysql://192.168.0.47:3306/hive?createDatabaseIfNotExist=true&useSSL=fal
Metastore Connection Driver :	 com.mysql.jdbc.Driver
Metastore connection User:	 root
Starting metastore schema initialization to 3.1.0
Initialization script hive-schema-3.1.0.mysql.sql
Initialization script completed
schemaTool completed

8、验证是否安装成功

输入hive命令，成功进入到hive的shell交互环境，就说明hive安装成功

[root@k8s-node3 ~]# hive 
which: no hbase in (/usr/local/nginx/sbin:/usr/local/php/bin:/usr/local/php/sbin:/home/software/spark-3.0.1-bin-hadoop2.7/bin:/home/software/spark-3.0.1-bin-hadoop2.7/sbin:/usr/java/jdk1.8.0_333-amd64//bin:/home/software/flink-1.14.4/bin/:/home/software/zookeeper-3.7.0/bin:/home/software/apache-hive-3.1.3-bin/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/home/software/hadoop-3.3.1//bin:/home/software/hadoop-3.3.1//sbin:/root/bin)
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/home/software/apache-hive-3.1.3-bin/lib/log4j-slf4j-impl-2.17.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/home/software/hadoop-3.3.1/share/hadoop/common/lib/slf4j-log4j12-1.7.30.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Hive Session ID = c2489767-0070-4a31-b304-cc9e2a2d7de5

Logging initialized using configuration in jar:file:/home/software/apache-hive-3.1.3-bin/lib/hive-common-3.1.3.jar!/hive-log4j2.properties Async: true
Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.
Hive Session ID = 65bb1a2e-8be6-4253-a484-d408d1f6fb01
hive>