【Hive】——安装部署

12 篇文章 0 订阅

1 MetaData(元数据)

在这里插入图片描述
在这里插入图片描述

2 MetaStore (元数据服务)

在这里插入图片描述

3 MetaStore配置方式

在这里插入图片描述
在这里插入图片描述

3.1 内嵌模式

在这里插入图片描述
在这里插入图片描述

3.2 本地模式

在这里插入图片描述
在这里插入图片描述

3.3 远程模式

在这里插入图片描述
在这里插入图片描述

4 安装前准备

在这里插入图片描述在这里插入图片描述

    <!-- 整合hive -->
    <property>
        <name>hadoop.proxyuser.root.hosts</name>
        <value>*</value>
    </property>
    <property>
        <name>hadoop.proxyuser.root.groups</name>
        <value>*</value>
    </property>

5 远程模式安装

5.1 下载

https://hive.apache.org/

5.2 解压并重命名

tar -zxvf apache-hive-3.1.2-bin.tar.gz -C /opt/module/
cd /opt/module/
mv mv apache-hive-3.1.2-bin hive

5.3 解决hadoop、hive 之间的guava版本差异问题

 cd /opt/module/hive/lib
 rm -f guava-19.0.jar
 cp /opt/module/hadoop-3.1.3/share/hadoop/common/lib/guava-27.0-jre.jar ./guava-27.0-jre.jar

5.4 添加环境变量

vi /etc/profile.d/my_env.sh
 #HIVE_HOME
 export HIVE_HOME=/opt/module/hive
 export PATH=$PATH:$HIVE_HOME/bin

source /etc/profile

5.5 hive-env.sh 修改Hive环境变量

cd /opt/module/hive/conf
mv hive-env.sh.template hive-env.sh
vim hive-env.sh

# Set HADOOP_HOME to point to a specific hadoop install directory
export HADOOP_HOME=/opt/module/hadoop-3.1.3

# Hive Configuration Directory can be controlled by:
export HIVE_CONF_DIR=/opt/module/hive/conf

# Folder containing extra libraries required for hive compilation/execution can be controlled by:
export HIVE_AUX_JARS_PATH=/opt/module/hive/lib

5.6 hive-log4j2.properties 日志配置

mkdir -P /opt/module/hive/datas
cd /opt/module/hive/conf
mv hive-log4j2.properties.template hive-log4j2.properties
vim hive-log4j2.properties

property.hive.log.dir = /opt/module/hive/datas

5.7 hive-site.xml 配置MateStore

添加了hive.metastore.uris 配置,则需要手动启动Matastore服务

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>

  <!-- 存储元数据mysql配置 -->
  <property>
    <name>javax.jdo.option.ConnectionURL</name>
    <value>
      jdbc:mysql://hadoop102:3306/hive?createDatabaseIfNotExist=true&amp;useUnicode=true&amp;useSSL=false&amp;characterEncoding=utf8</value>
    <description>
      JDBC connect string for a JDBC metastore.
      To use SSL to encrypt/authenticate the connection, provide database-specific SSL flag in the
      connection URL.
      For example, jdbc:postgresql://myhost/db?ssl=true for postgres database.
    </description>
  </property>
  <property>
    <name>javax.jdo.option.ConnectionDriverName</name>
    <value>com.mysql.cj.jdbc.Driver</value>
    <description>Driver class name for a JDBC metastore</description>
  </property>
  <property>
    <name>javax.jdo.option.ConnectionUserName</name>
    <value>root</value>
    <description>Username to use against metastore database</description>
  </property>
  <property>
    <name>javax.jdo.option.ConnectionPassword</name>
    <value>123456</value>
    <description>password to use against metastore database</description>
  </property>
  <!-- H2S运行绑定host -->
  <property>
    <name>hive.server2.thrift.bind.host</name>
    <value>hadoop102</value>
    <description>Bind host on which to run the HiveServer2 Thrift service.</description>
  </property>
  <!-- 远程模式部署metastore 服务地址 -->
  <property>
    <name>hive.metastore.uris</name>
    <value>thrift://hadoop102:9083</value>
    <description>Thrift URI for the remote metastore. Used by metastore client to connect to remote
      metastore.</description>
  </property>
  <!-- 关闭元数据存储授权 -->
  <property>
    <name>hive.metastore.event.db.notification.api.auth</name>
    <value>false</value>
    <description>
      Should metastore do authorization against database notification related APIs such as
      get_next_notification.
      If set to true, then only the superusers in proxy settings have the permission
    </description>
  </property>
  <!-- 关闭元数据存储版本的验证 -->
  <property>
    <name>hive.metastore.schema.verification</name>
    <value>true</value>
    <description>
      Enforce metastore schema version consistency.
      True: Verify that version information stored in is compatible with one from Hive jars. Also
      disable automatic
      schema migration attempt. Users are required to manually migrate schema after Hive upgrade
      which ensures
      proper metastore schema migration. (Default)
      False: Warn if the version information stored in metastore doesn't match with one from in Hive
      jars.
    </description>
  </property>
</configuration>

5.8 上传mysql-connector-java-5.1.27-bin.jar

基于mysql的版本上传jar

 /opt/module/hive/lib/mysql-connector-java-5.1.27-bin.jar

5.9 初始化Matedata

cd /opt/module/hive/bin
./schematool -dbType mysql  -initSchema  --verbose

3.5.10 启动MateStore脚本

vim hive_metastore.sh
#!/bin/bash
if [ $# -lt 1 ]; then
    echo "No Args Input..."
    exit
fi

case $1 in
"start")
    {
        echo "----------------- MetaStore start -----------------"
        nohup /opt/module/hive/bin/hive --service metastore >> /opt/module/hive/datas/metastore.out 2>&1 &
        echo "----------------- Hiveserver2 start -----------------"
        nohup /opt/module/hive/bin/hive --service hiveserver2 >> /opt/module/hive/datas/hiveserver2.out 2>&1 &
    }
    ;;
"stop")
    {
        echo "----------------- MetaStore stop -----------------"
        pidMetaStore=$(ps -ef | grep -v grep | grep "Dproc_metastore" | awk '{printf $2" "}')
        kill -9 ${pidMetaStore}
        echo "----------------- Hiveserver2 stop -----------------"
        pidHiveserver2=$(ps -ef | grep -v grep | grep "Dproc_hiveserver2" | awk '{printf $2" "}')
        kill -9 ${pidHiveserver2}
    }
    ;;
*)
    echo "Input Args Error..."
    ;;
esac

  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值