配置Hive
配置环境版本:
Centos 7
Hadoop 2.7.1
Mysql 5.6.48
Hive 2.3.6
Jdk 1.8.0_11
数据库驱动版本:Mysql-connector-java-5.1.47
附上hive国内镜像下载地址:
http://mirror.bit.edu.cn/apache/hive/
1.下载Hive安装包上传到/usr/local/hive,并解压
解压命令:
tar -zxvf 文件名.tar.gz
可将文件改名为hive2.3.6,方便配置
重命名命令格式:
mv 文件名 修改后的文件名
2.配置环境变量
vim /etc/profile
添加:
export HIVE_HOME=/usr/local/hive/hive2.3.6
export PATH=$PATH:$HIVE_HOME/bin
修改完后执行
source /etc/profile
查看hive环境变量是否配置成功
执行
hive -version
提示如下则配置成功
3.在usr/local/hive/hive2.3.6/conf/ 下创建hive-site.xml 文件:
复制hive-default.xml.template并更名为hive-site.xml
cp hive-default.xml.template hive-site.xml
4.创建hive-site.xml中HDFS相关设置对应目录
创建代码如下:
start-all.sh #开启hadoop服务
hadoop fs -mkdir -p /user/hive/warehouse
hadoop fs -mkdir -p /tmp/hive
hadoop fs -chmod -R 777 /user/hive/warehouse
hadoop fs -chmod -R 777 /tmp/hive
hadoop fs -ls /
5.修改hive-site.xml #在conf 目录下vim hive-site.xml
①查看hive目录下是否存在temp文件夹,若不存在则需要自行创建
创建temp文件夹命令如下:
mkdir temp
chmod -R 777 temp
②修改 hive-site.xml 文件
(vim hive-site.xml)
搜索关键字,修改配置
搜索技巧:
:/关键字
演示如下:
name 标签可以作为关键字
value 按自己情况进行修改
<property>
<name>hive.exec.local.scratchdir</name>
<value>/usr/local/hive/hive2.3.6/temp/root</value>
<description>Local scratch space for Hive jobs</description>
</property>
<property>
<name>hive.downloaded.resources.dir</name>
<value>/usr/local/hive/hive2.3.6/temp/${hive.session.id}_resources</value>
<description>Temporary local directory for added resources in the remote file system.</description>
</property>
<property>
<name>hive.server2.logging.operation.log.location</name>
<value>/usr/local/hive/hive2.3.6/temp/root/operation_logs</value>
<description>Top level directory where operation logs are stored if logging functionality is enabled</description>
</property>
<property>
<name>hive.querylog.location</name>
<value>/usr/local/hive/hive2.3.6/temp/root</value>
<description>Location of Hive run time structured log file</description>
</property>
<property>
<name>javax.jdo.option.ConnectionURL</name>
<value>jdbc:mysql://192.168.121.123:3306/hive?createDatabaseIfNotExist=true&characterEncoding=UTF-8</value>
<description>
JDBC connect string for a JDBC metastore.
To use SSL to encrypt/authenticate the connection, provide database-specific SSL flag in the connection URL.
For example, jdbc:postgresql://myhost/db?ssl=true for postgres database.
</description>
</property>
此处注意:(下面)
# 数据库的驱动类名称
# 新版本8.0版本的驱动为com.mysql.cj.jdbc.Driver
# 旧版本5.x版本的驱动为com.mysql.jdbc.Driver
# 本记录驱动版本为5.1.47
<property>
<name>javax.jdo.option.ConnectionDriverName</name>
<value>com.mysql.jdbc.Driver</value>
<description>Driver class name for a JDBC metastore</description>
</property>
<property>
<name>javax.jdo.option.ConnectionUserName</name>
<value>root</value> #mysql数据库用户名
<description>Username to use against metastore database</description>
</property>
<property>
<name>javax.jdo.option.ConnectionPassword</name>
<value>123456</value> # mysql密码
</property>
<property>
<name>hive.metastore.schema.verification</name>
<value>false</value>
<description>
Enforce metastore schema version consistency.
True: Verify that version information stored in is compatible with one from Hive jars. Also disable automatic
schema migration attempt. Users are required to manually migrate schema after Hive upgrade which ensures
proper metastore schema migration. (Default)
False: Warn if the version information stored in metastore doesn't match with one from in Hive jars.
</description>
</property>
两个都为false
6.配置hive-log4j2.properties
复制并更名hive-log4j2.properties.template为 hive-log4j2.properties文件:
命令
cp hive-log4j2.properties.template hive-log4j2.properties
vim hive-log4j2.properties
修改内容:
property.hive.log.dir = /usr/local/hive/hive2.3.6/temp/root
7.配置hive-env.sh文件
复制并更名hive-env.sh.template为 hive-env.sh文件:
cp hive-env.sh.template hive-env.sh
vim hive-env.sh
添加如下内容(在文本末端)根据自己情况填写
# Set HADOOP_HOME to point to a specific hadoop install directory
HADOOP_HOME=/home/software/hadoop-2.7.1 #hadoop 安装目录
# Hive Configuration Directory can be controlled by:
export HIVE_CONF_DIR=/usr/local/hive/hive2.3.6/conf #hive配置文件目录
# Folder containing extra libraries required for hive compilation/execution can be controlled by:
export HIVE_AUX_JARS_PATH=/usr/local/hive/hive2.3.6/lib #hive 依赖jar包目录
8.启动mysql 创建 hive 仓库
create database hive
9.MYSQL 权限设置
grant all privileges on *.* to 'root'@'%'identified by '密码'with grant option;
flush privileges; #刷新权限
10.上传mysql连接驱动到/usr/local/hive并解压,然后复制到hive目录中的lib中(如下图pwd路径)
注意:需要的是这个带-bin的jar包
步骤过程:
在/usr/local/hive/mysql-connector-java-5.1.47目录下输入
cp mysql-connector-java-5.1.47-bin.jar /usr/local/hive/hive2.3.6/lib
命令格式: cp 文件名 目标路径
完成后到 /usr/local/hive/hive2.3.6/lib 目录下执行
ll mysql-connector-java-5.1.47-bin.jar
查询jar包是否已经复制到指定目录
11.进入hive的bin目录进行初始化
初始化命令:
schematool -dbType mysql -initSchema
12.启动hive
直接输入
hive
可使用SQL命令创建数据库和查看数据库,确定是否能正常使用
😕😕😕
可能出现错误:
1.在hive中执行sql语句时报错:
SemanticException org.apache.hadoop.hive.ql.metadata.HiveException:
解决办法:
修改hive-site.xml文件添加如下配置
改false为true
<property>
<name>datanucleus.schema.autoCreateAll</name>
<value>true</value>
</property>
然后进入mysql,删除mysql中原来创建的hive表,重新初始化元数据
drop database hive;
create database hive;
在hive的bin目录下初始化元数据
schematool -dbType mysql -initSchema
😕