HIVE执行过程中java.lang.ArrayIndexOutOfBoundsException【待解决】

最新推荐文章于 2022-08-07 07:38:29 发布
最新推荐文章于 2022-08-07 07:38:29 发布
阅读量2.7k 收藏 1
点赞数
分类专栏： hive 文章标签： hive
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/zoe_ranxiaosu/article/details/116945415
版权

 

HIVE执行过程中java.lang.ArrayIndexOutOfBoundsException

表tmp_1

+-----------------------+-------------+
| tmp_1.id  | tmp_1.cl  | tmp_1.plat  |
+-----------------------+-------------+
| 293       | IOS_i01   | IOS         |
| 553       | IOS_i01   | IOS         |
| 559       | AND_a01   | AND         |
| 711       | AND_a01   | AND         |
+---------------+---------------------+

简单通过plat进行id关联

=============
示例1
=============
select 
a.plat,a.cnt,b.cnt
from 
(select
plat,count(*) cnt
from tmp_1
group by plat)a
left join
(select split(cl,'_')[0] as plat,count(*) cnt
from tmp_1 group by split(cl,'_')[0])b
on a.plat=b.plat;

 查询结果：
 +-------------+---------+---------+--+
| a.plat       |  a.cnt  |  b.cnt  |
+-------------+---------+---------+--+
| ALIPAY      | 371     | 371     |
| AND         | 783199  | 783199  |
| IOS         | 659319  | 659319  |
| WECHAT      | 2054    | 2054    |
+-------------+---------+---------+--+

执行正常，且查看执行过程

+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--+
|                                                                                                           Explain                                                                                                           |
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--+
| Plan not optimized by CBO.                                                                                                                                                                                                  |                                                                                                                                                                                                                            |
| Vertex dependency in root stage                                                                                                                                                                                             |
| Reducer 2 <- Map 1 (SIMPLE_EDGE), Reducer 4 (BROADCAST_EDGE)                                                                                                                                                                |
| Reducer 4 <- Map 3 (SIMPLE_EDGE)                                                                                                                                                                                            |                                                                                                                                                                                                                            |
| Stage-0                                                                                                                                                                                                                     |
|    Fetch Operator                                                                                                                                                                                                           |
|       limit:-1                                                                                                                                                                                                              |
|       Stage-1                                                                                                                                                                                                               |
|          Reducer 2                                                                                                                                                                                                          |
|          File Output Operator [FS_6313316]                                                                                                                                                                                  |
|             compressed:false                                                                                                                                                                                                |
|             Statistics:Num rows: 794718 Data size: 27599734 Basic stats: COMPLETE Column stats: NONE                                                                                                                        |
|             table:{"input format:":"org.apache.hadoop.mapred.TextInputFormat","output format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}  |
|             Select Operator [SEL_6313315]                                                                                                                                                                                   |
|                outputColumnNames:["_col0","_col1","_col2"]                                                                                                                                                                  |
|                Statistics:Num rows: 794718 Data size: 27599734 Basic stats: COMPLETE Column stats: NONE                                                                                                                     |
|                Map Join Operator [MAPJOIN_6313319]                                                                                                                                                                          |
|                |  condition map:[{"":"Left Outer Join0 to 1"}]                                                                                                                                                              |
|                |  HybridGraceHashJoin:true                                                                                                                                                                                  |
|                |  keys:{"Reducer 2":"_col0 (type: string)","Reducer 4":"_col0 (type: string)"}                                                                                                                              |
|                |  outputColumnNames:["_col0","_col1","_col3"]                                                                                                                                                               |
|                |  Statistics:Num rows: 794718 Data size: 27599734 Basic stats: COMPLETE Column stats: NONE                                                                                                                  |
|                |<-Reducer 4 [BROADCAST_EDGE]                                                                                                                                                                                |
|                |  Reduce Output Operator [RS_6313313]                                                                                                                                                                       |
|                |     key expressions:_col0 (type: string)                                                                                                                                                                   |
|                |     Map-reduce partition columns:_col0 (type: string)                                                                                                                                                      |
|                |     sort order:+                                                                                                                                                                                           |
|                |     Statistics:Num rows: 722471 Data size: 25090667 Basic stats: COMPLETE Column stats: NONE                                                                                                               |
|                |     value expressions:_col1 (type: bigint)                                                                                                                                                                 |
|                |     Group By Operator [GBY_6313310]                                                                                                                                                                        |
|                |     |  aggregations:["count(VALUE._col0)"]                                                                                                                                                                 |
|                |     |  keys:KEY._col0 (type: string)                                                                                                                                                                       |
|                |     |  outputColumnNames:["_col0","_col1"]                                                                                                                                                                 |
|                |     |  Statistics:Num rows: 722471 Data size: 25090667 Basic stats: COMPLETE Column stats: NONE                                                                                                            |
|                |     |<-Map 3 [SIMPLE_EDGE]                                                                                                                                                                                 |
|                |        Reduce Output Operator [RS_6313309]                                                                                                                                                                 |
|                |           key expressions:_col0 (type: string)                                                                                                                                                             |
|                |           Map-reduce partition columns:_col0 (type: string)                                                                                                                                                |
|                |           sort order:+                                                                                                                                                                                     |
|                |           Statistics:Num rows: 1444943 Data size: 50181369 Basic stats: COMPLETE Column stats: NONE                                                                                                        |
|                |           value expressions:_col1 (type: bigint)                                                                                                                                                           |
|                |           Group By Operator [GBY_6313308]                                                                                                                                                                  |
|                |              aggregations:["count()"]                                                                                                                                                                      |
|                |              keys:split(cl, '_')[0] (type: string)                                                                                                                                              |
|                |              outputColumnNames:["_col0","_col1"]                                                                                                                                                           |
|                |              Statistics:Num rows: 1444943 Data size: 50181369 Basic stats: COMPLETE Column stats: NONE                                                                                                     |
|                |              Select Operator [SEL_6313307]                                                                                                                                                                 |
|                |                 outputColumnNames:["cl"]                                                                                                                                                        |
|                |                 Statistics:Num rows: 1444943 Data size: 50181369 Basic stats: COMPLETE Column stats: NONE                                                                                                  |
|                |                 TableScan [TS_6313306]                                                                                                                                                                     |
|                |                    alias:tmp_1                                                                                                                                                                             |
|                |                    Statistics:Num rows: 1444943 Data size: 50181369 Basic stats: COMPLETE Column stats: NONE                                                                                               |
|                |<-Group By Operator [GBY_6313304]                                                                                                                                                                           |
|                   |  aggregations:["count(VALUE._col0)"]                                                                                                                                                                    |
|                   |  keys:KEY._col0 (type: string)                                                                                                                                                                          |
|                   |  outputColumnNames:["_col0","_col1"]                                                                                                                                                                    |
|                   |  Statistics:Num rows: 722471 Data size: 25090667 Basic stats: COMPLETE Column stats: NONE                                                                                                               |
|                   |<-Map 1 [SIMPLE_EDGE]                                                                                                                                                                                    |
|                      Reduce Output Operator [RS_6313303]                                                                                                                                                                    |
|                         key expressions:_col0 (type: string)                                                                                                                                                                |
|                         Map-reduce partition columns:_col0 (type: string)                                                                                                                                                   |
|                         sort order:+                                                                                                                                                                                        |
|                         Statistics:Num rows: 1444943 Data size: 50181369 Basic stats: COMPLETE Column stats: NONE                                                                                                           |
|                         value expressions:_col1 (type: bigint)                                                                                                                                                              |
|                         Group By Operator [GBY_6313302]                                                                                                                                                                     |
|                            aggregations:["count()"]                                                                                                                                                                         |
|                            keys:platform (type: string)                                                                                                                                                                     |
|                            outputColumnNames:["_col0","_col1"]                                                                                                                                                              |
|                            Statistics:Num rows: 1444943 Data size: 50181369 Basic stats: COMPLETE Column stats: NONE                                                                                                        |
|                            Select Operator [SEL_6313301]                                                                                                                                                                    |
|                               outputColumnNames:["plat"]                                                                                                                                                                |
|                               Statistics:Num rows: 1444943 Data size: 50181369 Basic stats: COMPLETE Column stats: NONE                                                                                                     |
|                               TableScan [TS_6313300]  alias:tmp_1 Statistics:Num rows: 1444943 Data size: 50181369 Basic stats: COMPLETE Column stats: NONE                                                                                                  |
|                                                                                                                                                                                                                             |
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--+

示例2 ，依然进行plat进行关联，只不过多加一个子句---会报错，越界的错误，很奇怪

=============
示例2
=============
select 
a.plat,a.cnt,b.cnt,c.cnt
from 
(select
plat,count(*) cnt
from tmp_db.tmp_1
group by plat)a
left join
(select split(cl,'_')[0] as plat,count(*) cnt
from tmp_db.tmp_1 group by split(cl,'_')[0])b
on a.plat=b.plat
left join
(select split(cl,'_')[0] as plat,count(*) cnt
from tmp_db.tmp_1 group by split(cl,'_')[0])c
on a.plat=c.plat
;

报错：
Caused by: java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row (tag=1) {"key":{"_col0":"ALIPAY"},"value":{"_col0":27}}
        at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:284)
        at org.apache.hadoop.hive.ql.exec.CommonMergeJoinOperator.fetchOneRow(CommonMergeJoinOperator.java:408)
        ... 26 more
Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row (tag=1) {"key":{"_col0":"ALIPAY"},"value":{"_col0":27}}
        at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:352)
        at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource.pushRecord(ReduceRecordSource.java:274)
        ... 27 more
Caused by: java.lang.ArrayIndexOutOfBoundsException: 1
        at org.apache.hadoop.hive.ql.exec.GroupByOperator.process(GroupByOperator.java:714)
        at org.apache.hadoop.hive.ql.exec.tez.ReduceRecordSource$GroupIterator.next(ReduceRecordSource.java:343)
        ... 28 more

示例2的执行计划

+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--+
|                                                                                                           Explain                                                                                                           |
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--+
| Plan not optimized by CBO due to missing statistics. Please check log for more details.                                                                                                                                     |
|                                                                                                                                                                                                                             |
| Vertex dependency in root stage                                                                                                                                                                                             |
| Reducer 2 <- Map 1 (SIMPLE_EDGE), Map 3 (SIMPLE_EDGE), Map 5 (SIMPLE_EDGE)                                                                                                                                                  |
|                                                                                                                                                                                                                             |
| Stage-0                                                                                                                                                                                                                     |
|    Fetch Operator                                                                                                                                                                                                           |
|       limit:-1                                                                                                                                                                                                              |
|       Stage-1                                                                                                                                                                                                               |
|          Reducer 2                                                                                                                                                                                                          |
|          File Output Operator [FS_6313426]                                                                                                                                                                                  |
|             compressed:false                                                                                                                                                                                                |
|             Statistics:Num rows: 1589436 Data size: 55199468 Basic stats: COMPLETE Column stats: NONE                                                                                                                       |
|             table:{"input format:":"org.apache.hadoop.mapred.TextInputFormat","output format:":"org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat","serde:":"org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe"}  |
|             Select Operator [SEL_6313425]                                                                                                                                                                                   |
|                outputColumnNames:["_col0","_col1","_col2","_col3"]                                                                                                                                                          |
|                Statistics:Num rows: 1589436 Data size: 55199468 Basic stats: COMPLETE Column stats: NONE                                                                                                                    |
|                Merge Join Operator [MERGEJOIN_6313431]                                                                                                                                                                      |
|                |  condition map:[{"":"Left Outer Join0 to 1"},{"":"Left Outer Join0 to 2"}]                                                                                                                                 |
|                |  keys:{"0":"_col0 (type: string)","1":"_col0 (type: string)","2":"_col0 (type: string)"}                                                                                                                   |
|                |  outputColumnNames:["_col0","_col1","_col3","_col5"]                                                                                                                                                       |
|                |  Statistics:Num rows: 1589436 Data size: 55199468 Basic stats: COMPLETE Column stats: NONE                                                                                                                 |
|                |                                                                                                                                                                                                            |
|                |<-Group By Operator [GBY_6313413]                                                                                                                                                                           |
|                |     aggregations:["count(VALUE._col0)"]                                                                                                                                                                    |
|                |     keys:KEY._col0 (type: string)                                                                                                                                                                          |
|                |     outputColumnNames:["_col0","_col1"]                                                                                                                                                                    |
|                |     Statistics:Num rows: 722471 Data size: 25090667 Basic stats: COMPLETE Column stats: NONE                                                                                                               |
|                |                                                                                                                                                                                                            |
|                |<-Group By Operator [GBY_6313419]                                                                                                                                                                           |
|                |     aggregations:["count(VALUE._col0)"]                                                                                                                                                                    |
|                |     keys:KEY._col0 (type: string)                                                                                                                                                                          |
|                |     outputColumnNames:["_col0","_col1"]                                                                                                                                                                    |
|                |     Statistics:Num rows: 722471 Data size: 25090667 Basic stats: COMPLETE Column stats: NONE                                                                                                               |
|                |<-Group By Operator [GBY_6313407]                                                                                                                                                                           |
|                   |  aggregations:["count(VALUE._col0)"]                                                                                                                                                                    |
|                   |  keys:KEY._col0 (type: string)                                                                                                                                                                          |
|                   |  outputColumnNames:["_col0","_col1"]                                                                                                                                                                    |
|                   |  Statistics:Num rows: 722471 Data size: 25090667 Basic stats: COMPLETE Column stats: NONE                                                                                                               |
|                   |<-Map 1 [SIMPLE_EDGE]                                                                                                                                                                                    |
|                   |  Reduce Output Operator [RS_6313406]                                                                                                                                                                    |
|                   |     key expressions:_col0 (type: string)                                                                                                                                                                |
|                   |     Map-reduce partition columns:_col0 (type: string)                                                                                                                                                   |
|                   |     sort order:+                                                                                                                                                                                        |
|                   |     Statistics:Num rows: 1444943 Data size: 50181369 Basic stats: COMPLETE Column stats: NONE                                                                                                           |
|                   |     value expressions:_col1 (type: bigint)                                                                                                                                                              |
|                   |     Group By Operator [GBY_6313405]                                                                                                                                                                     |
|                   |        aggregations:["count()"]                                                                                                                                                                         |
|                   |        keys:plat (type: string)                                                                                                                                                                     |
|                   |        outputColumnNames:["_col0","_col1"]                                                                                                                                                              |
|                   |        Statistics:Num rows: 1444943 Data size: 50181369 Basic stats: COMPLETE Column stats: NONE                                                                                                        |
|                   |        Select Operator [SEL_6313404]                                                                                                                                                                    |
|                   |           outputColumnNames:["plat"]                                                                                                                                                                |
|                   |           Statistics:Num rows: 1444943 Data size: 50181369 Basic stats: COMPLETE Column stats: NONE                                                                                                     |
|                   |           TableScan [TS_6313403]                                                                                                                                                                        |
|                   |              alias:tmp_1                                                                                                                                                                                |
|                   |              Statistics:Num rows: 1444943 Data size: 50181369 Basic stats: COMPLETE Column stats: NONE                                                                                                  |
|                   |<-Map 3 [SIMPLE_EDGE]                                                                                                                                                                                    |
|                   |  Reduce Output Operator [RS_6313412]                                                                                                                                                                    |
|                   |     key expressions:_col0 (type: string)                                                                                                                                                                |
|                   |     Map-reduce partition columns:_col0 (type: string)                                                                                                                                                   |
|                   |     sort order:+                                                                                                                                                                                        |
|                   |     Statistics:Num rows: 1444943 Data size: 50181369 Basic stats: COMPLETE Column stats: NONE                                                                                                           |
|                   |     value expressions:_col1 (type: bigint)                                                                                                                                                              |
|                   |     Group By Operator [GBY_6313411]                                                                                                                                                                     |
|                   |        aggregations:["count()"]                                                                                                                                                                         |
|                   |        keys:split(cl, '_')[0] (type: string)                                                                                                                                                 |
|                   |        outputColumnNames:["_col0","_col1"]                                                                                                                                                              |
|                   |        Statistics:Num rows: 1444943 Data size: 50181369 Basic stats: COMPLETE Column stats: NONE                                                                                                        |
|                   |        Select Operator [SEL_6313410]                                                                                                                                                                    |
|                   |           outputColumnNames:["cl"]                                                                                                                                                           |
|                   |           Statistics:Num rows: 1444943 Data size: 50181369 Basic stats: COMPLETE Column stats: NONE                                                                                                     |
|                   |           TableScan [TS_6313409]                                                                                                                                                                        |
|                   |              alias:tmp_1                                                                                                                                                                                |
|                   |              Statistics:Num rows: 1444943 Data size: 50181369 Basic stats: COMPLETE Column stats: NONE                                                                                                  |
|                   |<-Map 5 [SIMPLE_EDGE]                                                                                                                                                                                    |
|                      Reduce Output Operator [RS_6313418]                                                                                                                                                                    |
|                         key expressions:_col0 (type: string)                                                                                                                                                                |
|                         Map-reduce partition columns:_col0 (type: string)                                                                                                                                                   |
|                         sort order:+                                                                                                                                                                                        |
|                         Statistics:Num rows: 1444943 Data size: 50181369 Basic stats: COMPLETE Column stats: NONE                                                                                                           |
|                         value expressions:_col1 (type: bigint)                                                                                                                                                              |
|                         Group By Operator [GBY_6313417]                                                                                                                                                                     |
|                            aggregations:["count()"]                                                                                                                                                                         |
|                            keys:split(cl, '_')[0] (type: string)                                                                                                                                                 |
|                            outputColumnNames:["_col0","_col1"]                                                                                                                                                              |
|                            Statistics:Num rows: 1444943 Data size: 50181369 Basic stats: COMPLETE Column stats: NONE                                                                                                        |
|                            Select Operator [SEL_6313416]                                                                                                                                                                    |
|                               outputColumnNames:["cl"]                                                                                                                                                           |
|                               Statistics:Num rows: 1444943 Data size: 50181369 Basic stats: COMPLETE Column stats: NONE                                                                                                     |
|                               TableScan [TS_6313415]                                                                                                                                                                        |
|                                  alias:tmp_1                                                                                                                                                                                |
|                                  Statistics:Num rows: 1444943 Data size: 50181369 Basic stats: COMPLETE Column stats: NONE                                                                                                  |
|                                                                                                                                                                                                                             |
+-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+--+
冉小苏
关注
  • 0
    点赞
  • 踩
  • 1
    收藏
    觉得还不错? 一键收藏
  • 1
    评论
专栏目录
hive2.1.1中orc格式读取报数组越界错误解决方法
05-24
hive2.1.1orc格式读取报数组越界错误，替换jar包。hive-exec-2.1.1-cdh6.3.2.jar、hive-orc-2.1.1-cdh6.3.2.jar。分发各个服务器即可。
ArrayIndexOutOfBoundsException（数组越界异常）可能原因和解决办法
最新发布
Java开源程序猿
12-28 4784
（数组越界异常）通常在尝试访问数组中不存在的索引位置时抛出。确保在访问数组时，索引值在有效范围内，并在需要时进行适当的边界检查，可以有效地避免。
ArrayIndexOutOfBoundsException 异常报错原因及解决方案
weixin_46030002的博客
08-07 6万+
ArrayIndexOutOfBoundsException 异常报错原因及解决方案
hive中的 数组越界异常 IndexOutOfBoundsException
sinat_27047149的博客
07-21 1万+
java.lang.RuntimeException: Hive Runtime Error while closing operators at org.apache.hadoop.hive.ql.exec.ExecMapper.close(ExecMapper.java:226) at org.apache.hadoop.mapred.MapRunner.run(MapRunner.jav
FAILED: SemanticException org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeExcepti
01-07
错误详情 FAILED: SemanticException org.apache.hadoop.hive.ql.metadata.HiveException: java.lang.RuntimeException: Unable to instantiate org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient 解决方法 简单粗暴：在hvie-site.xml中关闭版本验证 hive.metastore.schema.verification false 目前这种方法只是强制性的关闭了hive的版本验
Hive多表连接异常，java.lang.ArrayIndexOutOfBounds ：140，官方Bug，在3.0.0版本已经被解决了
wunanliu的采坑日记
03-01 5106
后续官方解决方案：https://issues.apache.org/jira/browse/HIVE-14564 异常详细情况 2019-02-28 16:33:44,429 INFO [RMCommunicator Allocator] org.apache.hadoop.mapreduce.v2.app.rm.RMContainerAllocator: Got allocated ...
hive添加字段后的查询异常ArrayIndexOutOfBoundsException
qq_30868737的博客
06-15 2526
hive添加字段 alter table ods_wsp_dw_fact_point add columns (ods_datecreated STRING COMMENT ‘ods数据创建时间’); 客户端查询报错 hue端查询报错 Error while processing statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask 解决方案： 执行sql前，
Error解决：hive中的数组越界异常IndexOutOfBoundsException
小平子的专栏
12-12 7862
在spark中执行sql语句有时会出现IndexOutOfBoundsException的错误，如下： File "/opt/cloudera/parcels/SPARK2-2.3.0.cloudera2-1.cdh5.13.3.p0.316101/lib/spark2/python/lib/pyspark.zip/pyspark/sql/session.py", line 708, in sql...
关于执行SQL报错：java.lang.ArrayIndexOutOfBoundsException: 6
weixin_45901516的博客
04-14 4306
关于执行SQL报错：java.lang.ArrayIndexOutOfBoundsException: 6 由于工作需要，需创建一个表C，表数据由表A left join 表B生成，表A数据量大概300万行，表B数据量大概600万行左右，字段不多，但是脚本执行一直报错 一开始报错，查询日志提示信息为：return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask 百度查询具体原因要查看详细日志，打印日志发现提示内存溢出，但是2表的数据量不应该导
hive报错 spark_sparksql读取hive数据报错：java.lang.RuntimeException: serious problem
weixin_32616935的博客
02-23 670
问题：Caused by: java.util.concurrent.ExecutionException: java.lang.IndexOutOfBoundsException: Index: 0at java.util.concurrent.FutureTask.report(FutureTask.java:122)at java.util.concurrent.FutureTask.get...
apache-hive-2.3.9-bin.tar大数据HIVE.zip
10-08
打开压缩包得到apache-hive-2.3.9-bin.tar.gz
apache-hive-2.1.0-bin.tar.zip
05-04
apache-hive-2.1.0-bin
hive-jdbc-3.1.0.jar
08-23
hive jdbc 3.1.0 jar
java.util.concurrent.ExecutionException 错误解决。
Arog的博客
01-22 10万+
错误： java.util.concurrent.ExecutionException: org.apache.catalina.LifecycleException: Failed to start component [StandardEngine[Catalina].StandardHost[localhost].StandardContext[/recommender]] at jav
解决java.lang.IndexOutOfBoundsException: Index: 0, Size: 0的问题
热门推荐
cxws110的博客
09-15 15万+
出现这种问题是数组值没有取到，解决办法如下： 如：List orgList=sysOrgService.getByUserId(su.getUserId());             String orgName="";             if (orgList!=null&&orgList.size()>0) {                                 o
hive 分区表全量count(1)出现错误，单独查分区没问题
a312428841的博客
12-04 730
hive 分区表全量count(1)出现错误，单独查分区没问题 CDH5 中 hive1.1 版本，当修改表格的元数据信息后，使用ORC格式的文件与Hive的矢量化特性不兼容 ，需要先设置矢量化为false set hive.vectorized.execution.enabled=false; set hive.vectorized.execution.reduce.enabled=false; ...
hive ArrayIndexOutOfBoundsException
zhouzhoumi的专栏
03-09 1911
环境：hive on mr 异常： 2021-03-09 10:12:36,807 WARN [main] org.apache.hadoop.mapred.YarnChild: Exception running child : java.lang.RuntimeException: org.apache.hadoop.hive.ql.metadata.HiveException: Hive Runtime Error while processing row at org.apache.hado
flink写入orc文件到hive表，hive表读取报数组越界
Inkbamboo专栏
05-18 1818
组件版本: flink1.13.2 cdh6.3.2 hive2.1.1 问题描述: flink实时读取日志数据写入hdfs中，保存为orc格式文件。 flink写入文件格式: hive表建表语句: hive表查询时报: org.apache.hive.service.cli.HiveSQLException: java.io.IOException: java.lang.ArrayIndexOutOfBoundsException: 7 查看yarn日志: Caused
hivesql中常用函数
10-26
HiveSQL中常用的函数包括以下几类： 1. 聚合函数：如SUM、AVG、COUNT等，用于对数据进行统计分析。 2. 字符串函数：如CONCAT、SUBSTR、TRIM等，用于对字符串进行操作。 3. 数学函数：如ABS、CEIL、FLOOR等，用于对数字进行操作。 4. 日期函数：如YEAR、MONTH、DAY等，用于对日期进行操作。 5. 条件函数：如CASE、IF、COALESCE等，用于根据条件进行判断和处理。 6. 开窗函数：如ROW_NUMBER、RANK、DENSE_RANK等，用于对数据进行排序和分组。 7. 其他函数：如UDF、UDAF、UDTF等，用于自定义函数和聚合函数。 其中，COALESCE()函数在HiveSQL中也是常用的空值处理函数，而开窗函数在HiveSQL中也有类似于引用中提到的OVER子句的用法。

“相关推荐”对你有帮助么？

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论 1
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值