hive 服务 matastore服务

最新推荐文章于 2024-09-05 12:30:40 发布

澄续缘

最新推荐文章于 2024-09-05 12:30:40 发布

阅读量207

点赞数 2

分类专栏： Hive 文章标签： hive

本文链接：https://blog.csdn.net/weixuan_/article/details/138048573

版权

Hive 专栏收录该内容

11 篇文章 0 订阅

订阅专栏

本文详细描述了在Hadoop集群中部署HiveMetastore服务器，包括使用scp命令传输文件、配置metastore服务器和客户端，以及在不同模式下的行为。还讨论了Metastore客户端如何优先连接Metastore服务器而非JDBC。

摘要由CSDN通过智能技术生成

在这里插入图片描述

嵌入式模式
在这里插入图片描述

独立服务模式
在这里插入图片描述

在这里插入图片描述

部署

在这里插入图片描述

本次实验在Hadoop102部署metastore服务器，在Hadoop103部署metastore客户端
hadoop102

[atguigu@hadoop102 conf]$ scp -r /opt/module/apache-hive-3.1.3-bin/ hadoop103:/opt/module/
RELEASE_NOTES.txt                                                      100%  540   369.9KB/s   00:00    
LICENSE                                                                100%   20KB  12.3MB/s   00:00    
NOTICE                                                                 100%  230   213.0KB/s   00:00    
org.jruby-LICENSE                                                      100% 1023   679.5KB/s   00:00    
com.ibm.icu.icu4j-LICENSE                                              100%   19KB  12.1MB/s   00:00    
org.codehaus.janino-LICENSE                                            100% 1562     1.8MB/s   00:00    

启动metastore服务器
[atguigu@hadoop102 conf]$ nohup hive --service metastore &
[2] 12927
[atguigu@hadoop102 conf]$ nohup: 忽略输入并把输出追加到"nohup.out"

[atguigu@hadoop102 conf]$ jps -ml
7968 org.apache.hadoop.yarn.server.nodemanager.NodeManager
8800 org.apache.hadoop.util.RunJar /opt/module/apache-hive-3.1.3-bin/lib/hive-service-3.1.3.jar org.apache.hive.service.server.HiveServer2
7682 org.apache.hadoop.hdfs.server.datanode.DataNode
7507 org.apache.hadoop.hdfs.server.namenode.NameNode
2971 org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer
13068 sun.tools.jps.Jps -ml
12927 org.apache.hadoop.util.RunJar /opt/module/apache-hive-3.1.3-bin/lib/hive-metastore-3.1.3.jar org.apache.hadoop.hive.metastore.HiveMetaStore
[atguigu@hadoop102 conf]$

hadoop103


[atguigu@hadoop103 module]$ cd apache-hive-3.1.3-bin/conf/
[atguigu@hadoop103 conf]$ pwd
/opt/module/apache-hive-3.1.3-bin/conf
[atguigu@hadoop103 conf]$ vim hive-site.xml
[atguigu@hadoop103 conf]$ cat  hive-site.xml
<?xml version="1.0"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

<configuration>

	<property>
		<name>hive.metastore.warehouse.dir</name>
		<value>/user/hive/warehouse</value>
	</property>

<!-- 指定hiveserver2连接的host -->
	<property>
		<name>hive.server2.thrift.bind.host</name>
		<value>hadoop102</value>
	</property>

<!-- 指定hiveserver2连接的端口号 -->
	<property>
		<name>hive.server2.thrift.port</name>
		<value>10000</value>
	</property>

<!-- 指定metastore服务的地址 -->
	<property>
		<name>hive.metastore.uris</name>
		<value>thrift://hadoop102:9083</value>
	</property>

</configuration>

[atguigu@hadoop103 conf]$ cd ../
[atguigu@hadoop103 apache-hive-3.1.3-bin]$ pwd
/opt/module/apache-hive-3.1.3-bin

启动metastore客户端
[atguigu@hadoop103 apache-hive-3.1.3-bin]$ bin/hive
which: no hbase in (/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/module/jdk1.8.0_241/bin:/opt/module/hadoop-3.3.4/bin:/opt/module/hadoop-3.3.4/sbin:/home/atguigu/.local/bin:/home/atguigu/bin)
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/module/apache-hive-3.1.3-bin/lib/log4j-slf4j-impl-2.17.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/module/hadoop-3.3.4/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Hive Session ID = 674a898a-3637-410f-a8c2-89f83a518b37

Logging initialized using configuration in jar:file:/opt/module/apache-hive-3.1.3-bin/lib/hive-common-3.1.3.jar!/hive-log4j2.properties Async: true
Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.
Hive Session ID = 2414a6ef-37df-469d-a17d-3693944d2efc
hive> show databases;
OK
default
Time taken: 0.473 seconds, Fetched: 1 row(s)
hive> select * from stu;
OK
1	ss
1	ss
2	0421
3	0421-1
Time taken: 1.421 seconds, Fetched: 4 row(s)
hive> quit;
[atguigu@hadoop103 apache-hive-3.1.3-bin]$

关闭metastore服务

[atguigu@hadoop102 conf]$ jps -ml
7968 org.apache.hadoop.yarn.server.nodemanager.NodeManager
8800 org.apache.hadoop.util.RunJar /opt/module/apache-hive-3.1.3-bin/lib/hive-service-3.1.3.jar org.apache.hive.service.server.HiveServer2
7682 org.apache.hadoop.hdfs.server.datanode.DataNode
7507 org.apache.hadoop.hdfs.server.namenode.NameNode
13251 sun.tools.jps.Jps -ml
2971 org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer
12927 org.apache.hadoop.util.RunJar /opt/module/apache-hive-3.1.3-bin/lib/hive-metastore-3.1.3.jar org.apache.hadoop.hive.metastore.HiveMetaStore
[atguigu@hadoop102 conf]$ kill -9 12927
[atguigu@hadoop102 conf]$

如果metastore客户端同时配置jdbc和metastore服务器地址，metastore客户端只会访问服务器，而不是jdbc;

[atguigu@hadoop103 apache-hive-3.1.3-bin]$ bin/hive
which: no hbase in (/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/module/jdk1.8.0_241/bin:/opt/module/hadoop-3.3.4/bin:/opt/module/hadoop-3.3.4/sbin:/home/atguigu/.local/bin:/home/atguigu/bin)
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/module/apache-hive-3.1.3-bin/lib/log4j-slf4j-impl-2.17.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/module/hadoop-3.3.4/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Hive Session ID = 69183233-e1c6-4600-9a34-7175ab210632

Logging initialized using configuration in jar:file:/opt/module/apache-hive-3.1.3-bin/lib/hive-common-3.1.3.jar!/hive-log4j2.properties Async: true
Hive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.
hive> select * from stu;
FAILED: HiveException java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient
hive> quit;
[atguigu@hadoop103 apache-hive-3.1.3-bin]$

metastore服务器未开启，该设备hive客户端正常使用；

[atguigu@hadoop102 conf]$ jps -ml
7968 org.apache.hadoop.yarn.server.nodemanager.NodeManager
8800 org.apache.hadoop.util.RunJar /opt/module/apache-hive-3.1.3-bin/lib/hive-service-3.1.3.jar org.apache.hive.service.server.HiveServer2
7682 org.apache.hadoop.hdfs.server.datanode.DataNode
7507 org.apache.hadoop.hdfs.server.namenode.NameNode
13251 sun.tools.jps.Jps -ml
2971 org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer
12927 org.apache.hadoop.util.RunJar /opt/module/apache-hive-3.1.3-bin/lib/hive-metastore-3.1.3.jar org.apache.hadoop.hive.metastore.HiveMetaStore
[atguigu@hadoop102 conf]$ kill -9 12927
[atguigu@hadoop102 conf]$ hive
which: no hbase in (/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/module/jdk1.8.0_241/bin:/opt/module/hadoop-3.3.4/bin:/opt/module/hadoop-3.3.4/sbin:/opt/module/apache-hive-3.1.3-bin/bin:/home/atguigu/.local/bin:/home/atguigu/bin)
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/module/apache-hive-3.1.3-bin/lib/log4j-slf4j-impl-2.17.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/module/hadoop-3.3.4/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Hive Session ID = 9c2b5790-8088-40fb-af0a-0890945f050e

Logging initialized using configuration in jar:file:/opt/module/apache-hive-3.1.3-bin/lib/hive-common-3.1.3.jar!/hive-log4j2.properties Async: true
sel^H^HHive-on-MR is deprecated in Hive 2 and may not be available in the future versions. Consider using a different execution engine (i.e. spark, tez) or using Hive 1.X releases.
Hive Session ID = 6485836a-bc38-4c85-a437-75ccf8608d79
hive> select * from stu;
OK
1	ss
1	ss
2	0421
3	0421-1
Time taken: 1.442 seconds, Fetched: 4 row(s)
hive> quit;
[2]+  已杀死               nohup hive --service metastore
[atguigu@hadoop102 conf]$