hive使用技巧

澄续缘

于 2024-04-22 19:04:42 发布

阅读量286

点赞数 8

分类专栏： Hive 文章标签： hive hadoop 数据仓库

本文链接：https://blog.csdn.net/weixuan_/article/details/138049977

版权

Hive 专栏收录该内容

11 篇文章 0 订阅

订阅专栏

1、hive常用交互命令

在这里插入图片描述

hive -e 非交互式执行SQL

[atguigu@hadoop102 conf]$ hive -e "insert into stu values(1, 'abc')"
which: no hbase in (/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/module/jdk1.8.0_241/bin:/opt/module/hadoop-3.3.4/bin:/opt/module/hadoop-3.3.4/sbin:/opt/module/apache-hive-3.1.3-bin/bin:/home/atguigu/.local/bin:/home/atguigu/bin)
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/module/apache-hive-3.1.3-bin/lib/log4j-slf4j-impl-2.17.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/module/hadoop-3.3.4/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Hive Session ID = b28977c0-02cb-4f86-9803-73f2268ab6a0

Logging initialized using configuration in jar:file:/opt/module/apache-hive-3.1.3-bin/lib/hive-common-3.1.3.jar!/hive-log4j2.properties Async: true
Hive Session ID = 2650a002-5e5e-43cc-95fa-5348c3aa631e
Query ID = atguigu_20240421234049_c752127d-b8a8-4ca5-a750-00b078769ceb
Total jobs = 3
Launching Job 1 out of 3
Number of reduce tasks determined at compile time: 1
In order to change the average load for a reducer (in bytes):
  set hive.exec.reducers.bytes.per.reducer=<number>
In order to limit the maximum number of reducers:
  set hive.exec.reducers.max=<number>
In order to set a constant number of reducers:
  set mapreduce.job.reduces=<number>
Starting Job = job_1713696315107_0003, Tracking URL = http://hadoop103:8088/proxy/application_1713696315107_0003/
Kill Command = /opt/module/hadoop-3.3.4/bin/mapred job  -kill job_1713696315107_0003
Hadoop job information for Stage-1: number of mappers: 1; number of reducers: 1
2024-04-21 23:41:00,800 Stage-1 map = 0%,  reduce = 0%
2024-04-21 23:41:06,018 Stage-1 map = 100%,  reduce = 0%, Cumulative CPU 2.02 sec
2024-04-21 23:41:12,221 Stage-1 map = 100%,  reduce = 100%, Cumulative CPU 3.74 sec
MapReduce Total cumulative CPU time: 3 seconds 740 msec
Ended Job = job_1713696315107_0003
Stage-4 is selected by condition resolver.
Stage-3 is filtered out by condition resolver.
Stage-5 is filtered out by condition resolver.
Moving data to directory hdfs://hadoop102:8020/user/hive/warehouse/stu/.hive-staging_hive_2024-04-21_23-40-49_441_1318323531502153097-1/-ext-10000
Loading data to table default.stu
MapReduce Jobs Launched: 
Stage-Stage-1: Map: 1  Reduce: 1   Cumulative CPU: 3.74 sec   HDFS Read: 15228 HDFS Write: 236 SUCCESS
Total MapReduce CPU Time Spent: 3 seconds 740 msec
OK
Time taken: 25.011 seconds
[atguigu@hadoop102 conf]$ hive -e "select * from stu"
which: no hbase in (/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin:/opt/module/jdk1.8.0_241/bin:/opt/module/hadoop-3.3.4/bin:/opt/module/hadoop-3.3.4/sbin:/opt/module/apache-hive-3.1.3-bin/bin:/home/atguigu/.local/bin:/home/atguigu/bin)
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/opt/module/apache-hive-3.1.3-bin/lib/log4j-slf4j-impl-2.17.1.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/opt/module/hadoop-3.3.4/share/hadoop/common/lib/slf4j-reload4j-1.7.36.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.apache.logging.slf4j.Log4jLoggerFactory]
Hive Session ID = 6a96c066-1c7e-4d57-9b2b-f9a2d48ccace

Logging initialized using configuration in jar:file:/opt/module/apache-hive-3.1.3-bin/lib/hive-common-3.1.3.jar!/hive-log4j2.properties Async: true
Hive Session ID = 489a6445-08a9-4e03-85ad-937be55d7dc1
OK
1	ss
1	ss
2	0421
3	0421-1
1	abc
Time taken: 1.769 seconds, Fetched: 5 row(s)
[atguigu@hadoop102 conf]$

hive -f 非交互式执行SQL文件

在这里插入图片描述

2、hive参数配置方式

在这里插入图片描述

3、查看hive日志

[atguigu@hadoop102 conf]$ tail -10 /tmp/atguigu/hive.log 
2024-04-21T23:42:14,590  INFO [6a96c066-1c7e-4d57-9b2b-f9a2d48ccace main] conf.HiveConf: Using the default value passed in for log id: 6a96c066-1c7e-4d57-9b2b-f9a2d48ccace
2024-04-21T23:42:14,590  INFO [6a96c066-1c7e-4d57-9b2b-f9a2d48ccace main] CliDriver: Time taken: 1.769 seconds, Fetched: 5 row(s)
2024-04-21T23:42:14,590  INFO [6a96c066-1c7e-4d57-9b2b-f9a2d48ccace main] session.SessionState: Resetting thread name to  main
2024-04-21T23:42:14,590  INFO [main] conf.HiveConf: Using the default value passed in for log id: 6a96c066-1c7e-4d57-9b2b-f9a2d48ccace
2024-04-21T23:42:14,594  INFO [main] session.SessionState: Deleted directory: /tmp/hive/atguigu/6a96c066-1c7e-4d57-9b2b-f9a2d48ccace on fs with scheme hdfs
2024-04-21T23:42:14,602  INFO [main] session.SessionState: Deleted directory: /tmp/atguigu/6a96c066-1c7e-4d57-9b2b-f9a2d48ccace on fs with scheme file
2024-04-21T23:42:14,606  INFO [main] metastore.HiveMetaStore: 0: Cleaning up thread local RawStore...
2024-04-21T23:42:14,607  INFO [main] HiveMetaStore.audit: ugi=atguigu	ip=unknown-ip-addr	cmd=Cleaning up thread local RawStore...	
2024-04-21T23:42:14,607  INFO [main] metastore.HiveMetaStore: 0: Done cleaning up thread local RawStore
2024-04-21T23:42:14,607  INFO [main] HiveMetaStore.audit: ugi=atguigu	ip=unknown-ip-addr	cmd=Done cleaning up thread local RawStore	
[atguigu@hadoop102 conf]$