2024年大数据集群搭建之Linux安装Hive2(3)，2024年最新大数据开发开发社招面试经验

疯狂的石头。

于 2024-05-14 02:38:47 发布

阅读量934

点赞数 17

分类专栏：程序员文章标签：大数据面试学习

本文链接：https://blog.csdn.net/pasehelpme/article/details/138828879

版权

程序员专栏收录该内容

186 篇文章 1 订阅

订阅专栏

网上学习资料一大堆，但如果学到的知识不成体系，遇到问题时只是浅尝辄止，不再深入研究，那么很难做到真正的技术提升。

需要这份系统化资料的朋友，可以戳这里获取

一个人可以走的很快，但一群人才能走的更远！不论你是正从事IT行业的老鸟或是对IT行业感兴趣的新人，都欢迎加入我们的的圈子（技术交流、学习资源、职场吐槽、大厂内推、面试辅导），让我们一起学习成长！

1、hive环境变量配置

2、配置环境立即生效

四、Hive配置

1、hive-env.sh配置

2、hive-site.xml配置

五、初始化Hive

1、复制mysql jdbc驱动包到hive lib目录

2、MySQL创建用户并赋予权限

3、启动zk和hadoop集群

4、创建hive目录并赋权

5、初始化hive数据库

6、查看hive初始化的数据库

六、启动Hive

1、启动hive客户端

2、HDFS查看Hive目录

3、启动 HiveServer2 服务

4、查看 Hive 日志

5、查看 HiveServer2 webui

6、beeline连接Hive

七、Hive连接代码

1、hive jdbc连接

一、安装准备

二、解压Hive

1、解压文件

cd /usr/local/hadoop

tar zxpf apache-hive-2.3.2-bin.tar.gz

2、创建软链接

ln -s /usr/local/hadoop/apache-hive-2.3.2-bin /usr/local/hadoop/hive

三、环境变量配置

1、hive环境变量配置

vim /etc/profile

export HIVE_HOME=/usr/local/hadoop/hive

export PATH= $P A T H :$ HIVE_HOME/bin

2、配置环境立即生效

source /etc/profile

四、Hive配置

cd $HIVE_HOME/conf

touch hive-env.sh hive-site.xml

chmod +x hive-env.sh

1、hive-env.sh配置

export HADOOP_HEAPSIZE=4096

export JAVA_HOME=/usr/java/jdk1.8

export HADOOP_HOME=/usr/local/hadoop/hadoop

export HIVE_HOME=/usr/local/hadoop/hive

export HIVE_CONF_DIR=/usr/local/hadoop/hive/conf

export HBASE_HOME=/usr/local/hadoop/hbase

export SPARK_HOME=/usr/local/hadoop/spark

export ZOO_HOME=/usr/local/hadoop/zookeeper

2、hive-site.xml配置

<?xml version="1.0" encoding="UTF-8" standalone="no"?> <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

hive.metastore.warehouse.dir

/user/hive/warehouse

location of default database for the warehouse

hive.exec.local.scratchdir

/usr/local/hadoop/hive/tmp

Local scratch space for Hive jobs

hive.downloaded.resources.dir

/usr/local/hadoop/hive/tmp/resources

Temporary local directory for added resources in the remote file system.

hive.querylog.location

/user/hadoop/hive/logs

Location of Hive run time structured log file

hive.metastore.schema.verification

false

Enforce metastore schema version consistency.

True: Verify that version information stored in is compatible with one from Hive jars. Also disable automatic

schema migration attempt. Users are required to manually migrate schema after Hive upgrade which ensures

proper metastore schema migration. (Default)

False: Warn if the version information stored in metastore doesn’t match with one from in Hive jars.

hive.metastore.db.type

mysql

Expects one of [derby, oracle, mysql, mssql, postgres].

Type of database used by the metastore. Information schema & JDBCStorageHandler depend on it.

javax.jdo.option.ConnectionURL

jdbc:mysql://hadoop001:3306/hive?createDatabaseIfNotExist=true&useSSL=false

javax.jdo.option.ConnectionDriverName

com.mysql.jdbc.Driver

Driver class name for a JDBC metastore

javax.jdo.option.ConnectionUserName

hive

Username to use against metastore database

javax.jdo.option.ConnectionPassword

hive

Comma separated list of configuration options which should not be read by normal user like passwords

datanucleus.schema.autoCreateAll

true

Auto creates necessary schema on a startup if one doesn’t exist. Set this to false, after creating it once.To enable auto create also set hive.metastore.schema.verification=false. Auto creation is not recommended for production use cases, run schematool command instead.

hive.server2.thrift.bind.host

hadoop001

Bind host on which to run the HiveServer2 Thrift service.

hive.server2.thrift.port

10000

Port number of HiveServer2 Thrift interface when hive.server2.transport.mode is ‘binary’.

hive.metastore.uris

Thrift URI for the remote metastore. Used by metastore client to connect to remote metastore.

hive.server2.logging.operation.log.location

/usr/local/hadoop/hive/tmp/operation_logs

Top level directory where operation logs are stored if logging functionality is enabled

hive.server2.webui.host