Hive报错：Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask MapReduce

最新推荐文章于 2023-12-18 11:24:36 发布
最新推荐文章于 2023-12-18 11:24:36 发布
阅读量3.4k 收藏 10
点赞数 5
分类专栏： 日常改BUG hive 文章标签： hive 大数据
版权声明：本文为博主原创文章，遵循 CC 4.0 BY-SA 版权协议，转载请附上原文出处链接和本声明。
本文链接：https://blog.csdn.net/dazuo_001/article/details/117000437
版权

最近在做hive练习题时使用hive查询报错

Hadoop job information for Stage-3: number of mappers: 0; number of reducers: 0
2021-05-18 19:23:54,652 Stage-3 map = 0%,  reduce = 0%

Ended Job = job_1621330130219_0008 with errors
Error during job, obtaining debugging information...
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask

MapReduce Jobs Launched: 
Stage-Stage-3:  HDFS Read: 0 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 0 msec**

在这里插入图片描述

首先检查了sql语句以及hive配置都没有问题，
1.查看虚拟机的内存
在这里插入图片描述以为是内存小了，后又重新增加了内存后还是报同样的错
在这里插入图片描述
2.那么猜测可能原因是集群中各个节点的时间不同步，查看各节点的时间发现不同步，随后修改同步后，再次运行hql语句然而还是报错？？？咦，这就奇怪了？

查看hive日志(安装hive时没有设置过日志地址的话，会默认在系统的tmp目录中)使用root用户时，在/tmp/root/ 目录下），查看最新日志：

2021-05-21 14:35:30,190 INFO  [main]: ql.Driver (Driver.java:execute(1328)) - Starting command(queryId=root_20210521143529_36beccba-cbb9-40c3-ad43-10d3c73c8a03): select sc1.sid,sc1.score score1,sc2.score score2
from sc sc1 
join sc sc2 on sc1.sid=sc2.sid
where sc1.cid = 1 and sc2.cid = 2 and sc1.score>sc2.score
2021-05-21 14:35:30,190 INFO  [main]: ql.Driver (SessionState.java:printInfo(951)) - Query ID = root_20210521143529_36beccba-cbb9-40c3-ad43-10d3c73c8a03
2021-05-21 14:35:30,190 INFO  [main]: ql.Driver (SessionState.java:printInfo(951)) - Total jobs = 1
2021-05-21 14:35:30,191 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(148)) - </PERFLOG method=TimeToSubmit start=1621578929684 end=1621578930191 duration=507 from=org.apache.hadoop.hive.ql.Driver>
2021-05-21 14:35:30,191 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(121)) - <PERFLOG method=runTasks from=org.apache.hadoop.hive.ql.Driver>
2021-05-21 14:35:30,191 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(121)) - <PERFLOG method=task.MAPREDLOCAL.Stage-4 from=org.apache.hadoop.hive.ql.Driver>
2021-05-21 14:35:30,196 INFO  [main]: ql.Driver (Driver.java:launchTask(1651)) - Starting task [Stage-4:MAPREDLOCAL] in serial mode
2021-05-21 14:35:30,201 INFO  [main]: mr.MapredLocalTask (MapredLocalTask.java:executeInChildVM(157)) - Generating plan file file:/usr/soft/hive/hive-1.2.1/tmp/a3188886-f1af-440a-8eae-1136908ec0be/hive_2021-05-21_14-35-29_689_3311147117394005582-1/-local-10005/plan.xml
2021-05-21 14:35:30,216 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(121)) - <PERFLOG method=serializePlan from=org.apache.hadoop.hive.ql.exec.Utilities>
2021-05-21 14:35:30,216 INFO  [main]: exec.Utilities (Utilities.java:serializePlan(938)) - Serializing MapredLocalWork via kryo
2021-05-21 14:35:30,235 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(148)) - </PERFLOG method=serializePlan start=1621578930216 end=1621578930235 duration=19 from=org.apache.hadoop.hive.ql.exec.Utilities>
2021-05-21 14:35:30,439 INFO  [main]: mr.MapredLocalTask (MapredLocalTask.java:executeInChildVM(286)) - Executing: /usr/soft/hadoop/hadoop-2.6.5/bin/hadoop jar /usr/soft/hive/hive-1.2.1/lib/hive-exec-1.2.1.jar org.apache.hadoop.hive.ql.exec.mr.ExecDriver -localtask -plan file:/usr/soft/hive/hive-1.2.1/tmp/a3188886-f1af-440a-8eae-1136908ec0be/hive_2021-05-21_14-35-29_689_3311147117394005582-1/-local-10005/plan.xml   -jobconffile file:/usr/soft/hive/hive-1.2.1/tmp/a3188886-f1af-440a-8eae-1136908ec0be/hive_2021-05-21_14-35-29_689_3311147117394005582-1/-local-10006/jobconf.xml
2021-05-21 14:35:38,142 INFO  [main]: exec.Task (SessionState.java:printInfo(951)) - Execution completed successfully
2021-05-21 14:35:38,142 INFO  [main]: exec.Task (SessionState.java:printInfo(951)) - MapredLocal task succeeded
2021-05-21 14:35:38,142 INFO  [main]: mr.MapredLocalTask (MapredLocalTask.java:executeInChildVM(311)) - Execution completed successfully
2021-05-21 14:35:38,142 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(121)) - <PERFLOG method=task.MAPRED.Stage-3 from=org.apache.hadoop.hive.ql.Driver>
2021-05-21 14:35:38,143 INFO  [main]: ql.Driver (SessionState.java:printInfo(951)) - Launching Job 1 out of 1
2021-05-21 14:35:38,143 INFO  [main]: ql.Driver (Driver.java:launchTask(1651)) - Starting task [Stage-3:MAPRED] in serial mode
2021-05-21 14:35:38,147 INFO  [main]: exec.Task (SessionState.java:printInfo(951)) - Number of reduce tasks is set to 0 since there's no reduce operator
2021-05-21 14:35:38,147 INFO  [main]: ql.Context (Context.java:getMRScratchDir(330)) - New scratch dir is hdfs://master18402320:9000/tmp/hive/root/a3188886-f1af-440a-8eae-1136908ec0be/hive_2021-05-21_14-35-29_689_3311147117394005582-1
2021-05-21 14:35:38,153 INFO  [main]: mr.ExecDriver (ExecDriver.java:execute(288)) - Using org.apache.hadoop.hive.ql.io.CombineHiveInputFormat
2021-05-21 14:35:38,207 INFO  [main]: mr.ExecDriver (ExecDriver.java:execute(356)) - Archive 1 hash table files to file:/usr/soft/hive/hive-1.2.1/tmp/a3188886-f1af-440a-8eae-1136908ec0be/hive_2021-05-21_14-35-29_689_3311147117394005582-1/-local-10003/HashTable-Stage-3/Stage-3.tar.gz
2021-05-21 14:35:38,228 INFO  [main]: mr.ExecDriver (ExecDriver.java:execute(363)) - Upload 1 archive file  fromfile:/usr/soft/hive/hive-1.2.1/tmp/a3188886-f1af-440a-8eae-1136908ec0be/hive_2021-05-21_14-35-29_689_3311147117394005582-1/-local-10003/HashTable-Stage-3/Stage-3.tar.gz to: hdfs://master18402320:9000/tmp/hive/root/a3188886-f1af-440a-8eae-1136908ec0be/hive_2021-05-21_14-35-29_689_3311147117394005582-1/-mr-10004/HashTable-Stage-3/Stage-3.tar.gz
2021-05-21 14:35:38,228 INFO  [main]: mr.ExecDriver (ExecDriver.java:execute(368)) - Add 1 archive file to distributed cache. Archive file: hdfs://master18402320:9000/tmp/hive/root/a3188886-f1af-440a-8eae-1136908ec0be/hive_2021-05-21_14-35-29_689_3311147117394005582-1/-mr-10004/HashTable-Stage-3/Stage-3.tar.gz
2021-05-21 14:35:38,228 INFO  [main]: exec.Utilities (Utilities.java:getInputPaths(3397)) - Processing alias sc2
2021-05-21 14:35:38,228 INFO  [main]: exec.Utilities (Utilities.java:getInputPaths(3414)) - Adding input file hdfs://master18402320:9000/user/hive/warehouse/db_hive_test01.db/sc
2021-05-21 14:35:38,228 INFO  [main]: exec.Utilities (Utilities.java:isEmptyPath(2694)) - Content Summary hdfs://master18402320:9000/user/hive/warehouse/db_hive_test01.db/sclength: 162 num files: 1 num directories: 1
2021-05-21 14:35:38,229 INFO  [main]: ql.Context (Context.java:getMRScratchDir(330)) - New scratch dir is hdfs://master18402320:9000/tmp/hive/root/a3188886-f1af-440a-8eae-1136908ec0be/hive_2021-05-21_14-35-29_689_3311147117394005582-1
2021-05-21 14:35:38,235 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(121)) - <PERFLOG method=serializePlan from=org.apache.hadoop.hive.ql.exec.Utilities>
2021-05-21 14:35:38,236 INFO  [main]: exec.Utilities (Utilities.java:serializePlan(938)) - Serializing MapWork via kryo
2021-05-21 14:35:38,281 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(148)) - </PERFLOG method=serializePlan start=1621578938235 end=1621578938281 duration=46 from=org.apache.hadoop.hive.ql.exec.Utilities>
2021-05-21 14:35:38,282 ERROR [main]: mr.ExecDriver (ExecDriver.java:execute(400)) - yarn
2021-05-21 14:35:38,324 INFO  [main]: client.RMProxy (RMProxy.java:createRMProxy(98)) - Connecting to ResourceManager at master18402320/192.168.29.6:8032
2021-05-21 14:35:38,374 INFO  [main]: client.RMProxy (RMProxy.java:createRMProxy(98)) - Connecting to ResourceManager at master18402320/192.168.29.6:8032
2021-05-21 14:35:38,377 INFO  [main]: exec.Utilities (Utilities.java:getBaseWork(390)) - PLAN PATH = hdfs://master18402320:9000/tmp/hive/root/a3188886-f1af-440a-8eae-1136908ec0be/hive_2021-05-21_14-35-29_689_3311147117394005582-1/-mr-10008/2f8a9cc2-16cf-4426-8a9c-a175b89eaa53/map.xml
2021-05-21 14:35:38,377 INFO  [main]: exec.Utilities (Utilities.java:getBaseWork(390)) - PLAN PATH = hdfs://master18402320:9000/tmp/hive/root/a3188886-f1af-440a-8eae-1136908ec0be/hive_2021-05-21_14-35-29_689_3311147117394005582-1/-mr-10008/2f8a9cc2-16cf-4426-8a9c-a175b89eaa53/reduce.xml
2021-05-21 14:35:38,377 INFO  [main]: exec.Utilities (Utilities.java:getBaseWork(400)) - ***************non-local mode***************
2021-05-21 14:35:38,377 INFO  [main]: exec.Utilities (Utilities.java:getBaseWork(404)) - local path = hdfs://master18402320:9000/tmp/hive/root/a3188886-f1af-440a-8eae-1136908ec0be/hive_2021-05-21_14-35-29_689_3311147117394005582-1/-mr-10008/2f8a9cc2-16cf-4426-8a9c-a175b89eaa53/reduce.xml
2021-05-21 14:35:38,377 INFO  [main]: exec.Utilities (Utilities.java:getBaseWork(416)) - Open file to read in plan: hdfs://master18402320:9000/tmp/hive/root/a3188886-f1af-440a-8eae-1136908ec0be/hive_2021-05-21_14-35-29_689_3311147117394005582-1/-mr-10008/2f8a9cc2-16cf-4426-8a9c-a175b89eaa53/reduce.xml
2021-05-21 14:35:38,402 INFO  [main]: exec.Utilities (Utilities.java:getBaseWork(456)) - File not found: File does not exist: /tmp/hive/root/a3188886-f1af-440a-8eae-1136908ec0be/hive_2021-05-21_14-35-29_689_3311147117394005582-1/-mr-10008/2f8a9cc2-16cf-4426-8a9c-a175b89eaa53/reduce.xml
	at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:66)
	at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:56)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsUpdateTimes(FSNamesystem.java:1893)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsInt(FSNamesystem.java:1834)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1814)
	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1786)
	at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:552)
	at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:362)
	at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
	at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:619)
	at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:975)
	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2040)
	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2036)
	at java.security.AccessController.doPrivileged(Native Method)
	at javax.security.auth.Subject.doAs(Subject.java:422)
	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1692)
	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2034)

2021-05-21 14:35:38,402 INFO  [main]: exec.Utilities (Utilities.java:getBaseWork(457)) - No plan file found: hdfs://master18402320:9000/tmp/hive/root/a3188886-f1af-440a-8eae-1136908ec0be/hive_2021-05-21_14-35-29_689_3311147117394005582-1/-mr-10008/2f8a9cc2-16cf-4426-8a9c-a175b89eaa53/reduce.xml
2021-05-21 14:35:38,414 WARN  [main]: mapreduce.JobResourceUploader (JobResourceUploader.java:uploadFiles(64)) - Hadoop command-line option parsing not performed. Implement the Tool interface and execute your application with ToolRunner to remedy this.
2021-05-21 14:35:38,796 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(121)) - <PERFLOG method=getSplits from=org.apache.hadoop.hive.ql.io.CombineHiveInputFormat>
2021-05-21 14:35:38,796 INFO  [main]: exec.Utilities (Utilities.java:getBaseWork(390)) - PLAN PATH = hdfs://master18402320:9000/tmp/hive/root/a3188886-f1af-440a-8eae-1136908ec0be/hive_2021-05-21_14-35-29_689_3311147117394005582-1/-mr-10008/2f8a9cc2-16cf-4426-8a9c-a175b89eaa53/map.xml
2021-05-21 14:35:38,796 INFO  [main]: io.CombineHiveInputFormat (CombineHiveInputFormat.java:getSplits(517)) - Total number of paths: 1, launching 1 threads to check non-combinable ones.
2021-05-21 14:35:38,803 INFO  [main]: io.CombineHiveInputFormat (CombineHiveInputFormat.java:getCombineSplits(439)) - CombineHiveInputSplit creating pool for hdfs://master18402320:9000/user/hive/warehouse/db_hive_test01.db/sc; using filter path hdfs://master18402320:9000/user/hive/warehouse/db_hive_test01.db/sc
2021-05-21 14:35:38,810 INFO  [main]: input.FileInputFormat (FileInputFormat.java:listStatus(281)) - Total input paths to process : 1
2021-05-21 14:35:38,810 INFO  [main]: input.CombineFileInputFormat (CombineFileInputFormat.java:createSplits(413)) - DEBUG: Terminated node allocation with : CompletedNodes: 1, size left: 0
2021-05-21 14:35:38,811 INFO  [main]: io.CombineHiveInputFormat (CombineHiveInputFormat.java:getCombineSplits(494)) - number of splits 1
2021-05-21 14:35:38,813 INFO  [main]: io.CombineHiveInputFormat (CombineHiveInputFormat.java:getSplits(587)) - Number of all splits 1
2021-05-21 14:35:38,814 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(148)) - </PERFLOG method=getSplits start=1621578938796 end=1621578938814 duration=18 from=org.apache.hadoop.hive.ql.io.CombineHiveInputFormat>
2021-05-21 14:35:38,863 INFO  [main]: mapreduce.JobSubmitter (JobSubmitter.java:submitJobInternal(199)) - number of splits:1
2021-05-21 14:35:38,898 INFO  [main]: mapreduce.JobSubmitter (JobSubmitter.java:printTokens(288)) - Submitting tokens for job: job_1621577750352_0002
2021-05-21 14:35:38,931 INFO  [main]: impl.YarnClientImpl (YarnClientImpl.java:submitApplication(251)) - Submitted application application_1621577750352_0002
2021-05-21 14:35:38,934 INFO  [main]: mapreduce.Job (Job.java:submit(1301)) - The url to track the job: http://master18402320:8088/proxy/application_1621577750352_0002/
2021-05-21 14:35:38,935 INFO  [main]: exec.Task (SessionState.java:printInfo(951)) - Starting Job = job_1621577750352_0002, Tracking URL = http://master18402320:8088/proxy/application_1621577750352_0002/
2021-05-21 14:35:38,935 INFO  [main]: exec.Task (SessionState.java:printInfo(951)) - Kill Command = /usr/soft/hadoop/hadoop-2.6.5/bin/hadoop job  -kill job_1621577750352_0002
2021-05-21 14:35:50,451 INFO  [main]: exec.Task (SessionState.java:printInfo(951)) - Hadoop job information for Stage-3: number of mappers: 1; number of reducers: 0
2021-05-21 14:35:50,554 WARN  [main]: mapreduce.Counters (AbstractCounters.java:getGroup(234)) - Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead
2021-05-21 14:35:50,554 INFO  [main]: exec.Task (SessionState.java:printInfo(951)) - 2021-05-21 14:35:50,553 Stage-3 map = 0%,  reduce = 0%
2021-05-21 14:36:06,488 WARN  [main]: mapreduce.Counters (AbstractCounters.java:getGroup(234)) - Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead
2021-05-21 14:36:06,490 ERROR [main]: exec.Task (SessionState.java:printError(960)) - Ended Job = job_1621577750352_0002 with errors
2021-05-21 14:36:06,491 ERROR [Thread-72]: exec.Task (SessionState.java:printError(960)) - Error during job, obtaining debugging information...
2021-05-21 14:36:06,541 INFO  [main]: impl.YarnClientImpl (YarnClientImpl.java:killApplication(364)) - Killed application application_1621577750352_0002
2021-05-21 14:36:06,543 ERROR [main]: ql.Driver (SessionState.java:printError(960)) - FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
2021-05-21 14:36:06,543 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(148)) - </PERFLOG method=Driver.execute start=1621578930190 end=1621578966543 duration=36353 from=org.apache.hadoop.hive.ql.Driver>
2021-05-21 14:36:06,543 INFO  [main]: ql.Driver (SessionState.java:printInfo(951)) - MapReduce Jobs Launched: 
2021-05-21 14:36:06,544 WARN  [main]: mapreduce.Counters (AbstractCounters.java:getGroup(234)) - Group FileSystemCounters is deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead
2021-05-21 14:36:06,544 INFO  [main]: ql.Driver (SessionState.java:printInfo(951)) - Stage-Stage-3: Map: 1   HDFS Read: 0 HDFS Write: 0 FAIL
2021-05-21 14:36:06,544 INFO  [main]: ql.Driver (SessionState.java:printInfo(951)) - Total MapReduce CPU Time Spent: 0 msec
2021-05-21 14:36:06,545 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(121)) - <PERFLOG method=releaseLocks from=org.apache.hadoop.hive.ql.Driver>
2021-05-21 14:36:06,545 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(148)) - </PERFLOG method=releaseLocks start=1621578966545 end=1621578966545 duration=0 from=org.apache.hadoop.hive.ql.Driver>
2021-05-21 14:36:06,545 INFO  [main]: exec.ListSinkOperator (Operator.java:close(612)) - 15 finished. closing... 
2021-05-21 14:36:06,545 INFO  [main]: exec.ListSinkOperator (Operator.java:close(634)) - 15 Close done
2021-05-21 14:36:06,577 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogBegin(121)) - <PERFLOG method=releaseLocks from=org.apache.hadoop.hive.ql.Driver>
2021-05-21 14:36:06,577 INFO  [main]: log.PerfLogger (PerfLogger.java:PerfLogEnd(148)) - </PERFLOG method=releaseLocks start=1621578966577 end=1621578966577 duration=0 from=org.apache.hadoop.hive.ql.Driver>

查看日志发现是mapreduce没有正常运行，那么又去检查了一遍Hadoop集群搭建的整个配置文件，没有找到错误，所有的配制都是正确的。奇怪，那这是什么神马问题？？？随后就去跑一下自带的jar包看看

./bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.6.5.jar pi 3 10

果然也跑不起来。那这就奇怪了哈，随后检查hadoop yarn 上各个节点的applicationID日志 发现是无法识别我的节点名，原来是因为我的节点名里面有一个下划线，系统无法识别导致的。修改过后，就正常的跑起来了。hive查询语句也可以跑起来了。
在这里插入图片描述

北慕辰
关注
专栏目录
Hive错误之 Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask错误分析
xiaohu21的博客
09-04 4万+
Hive Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask错误分析 1. 背景 使用hive过程中，遇到问题有时候信息不是很足够，这时候就需要去想办法查看错误信息。而这类服务器框架的错误信息一般都存放在日志中 hive的日志不设置的话，默认存放在/tmp下当前用户所在目录，本文演示是root用户，所以文件存放在tmp文件夹下的root目录下/tmp/root 已经按照日期和时间进行存储，查看最
【Hive报错】FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
debimeng
12-13 2255
【场景】 在hive3.1版本下执行以下语句报错 --加载数据到桶表u_demo_bk中 insert into table u_tab_bk_p select * from u_tab; 可是插入到普通表和分区表的语句是能正常执行的。 【原因】 最后才知道是因为资源不足的问题，可是这数据量又不大，有可能hive的机制有关，本来它的场景就是为了解决大数据的问题。 【解决】 最后在主节点修改了yarn-site.xml文件，添加一个参数，即：关闭虚拟内存的检测，最后完整的内容如下： <co
hive报错Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
X_PENG_7的博客
05-14 1081
第三步：schematool -dbType mysql -initSchema （初始化MySQL数据库，重新指定数据库）第二步：重新创建metastore(create database metastore;) 退出quit;第一步：在mysql删掉metastore(drop database metastore;第四步：再次进入hive 创建表，添加数据。
hive报错FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
m1913843179的博客
11-20 1万+
这里写自定义目录标题新的改变功能快捷键合理的创建标题，有助于目录的生成如何改变文本的样式插入链接与图片如何插入一段漂亮的代码片生成一个适合你的列表创建一个表格设定内容居中、居左、居右SmartyPants创建一个自定义列表如何创建一个注脚注释也是必不可少的KaTeX数学公式新的甘特图功能，丰富你的文章UML 图表FLowchart流程图导出与导入导出导入 问题一： 你好！ 这是你第一次使用 Markdown编辑器 所展示的欢迎页。如果你想学习如何使用Markdown编辑器, 可以仔细阅读这篇文章，了解一下M
Execution Error,return code 2 from org.apache.haddop.hive.ql.exec.mr.MapRedTask解决办法
chongjiapi1753的博客
10-22 1012
insert overwrite table XX 1.建表后修改过表结构（增删字段等）后，重新加载数据时报return code2错，元数据和数据文件不匹配导致 解决办法：重建一张新表； 2.解决办法：修改表结构时加cascade 3.若排查定位是内存不足，则到资源充足的库新建...
hive问题-return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
erainm
09-23 9044
执行hive sql时遇到问题： FAILED:Execution Error,return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask 这个其实一般是个hive内部错误，处理这个问题就需要找日志了； 找日志 hive.log(比较简略) hive.log ===> 缺省情况下 /tmp/root/hive.log (不然就在你hive-site.conf配置的日志路径里) 我这里在/tmp/root目录下 去里面看了日志，没
在hive插入数据时出现“Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask”报错
m0_74756715的博客
11-04 8496
运行Tez时检查到用过多内存而被NodeManager杀死进程问题 设置hive的执行引擎
hive报错,FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
worldair的博客
12-18 864
首先,将hadoop下的yarn-site.xml文件复制到hive目录下的conf目录中,然后检查yarn-site.xml配置。yarn.application.classpath的值使用hadoop classpath命令获取然后复制进去,这个是最主要的。其他文章的方法千篇一律,但是弄了之后都没有用。然后那个select查询命令就能执行陈功了。网上找到的解决方法在我这里都没有用。弄完之后重启hive。
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
weixin_43579015的博客
05-11 8035
hiveserver2中运行命令报错 Execution Error 解决办法、 1、调大虚拟机内存，单台节点至少1G可用，调大虚拟机内存不需要关机，调小需要关机，如果上限只有XG，不能调大，关机再调，再调节MapReduce单个任务分配的内存分配的内存大小 set mapreduce.map.memory.mb=2048; set mapreduce.reduce.memory.mb=2048; 2、yarn-env。sh增加JAVA_HOME(注意发送到另外两台服务器) export JAVA_HOM
hive sql语句报错：FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTas
OwnYoung的博客
01-03 6690
报错内容：FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTas 原因：我是创建新表导致，超过58w条记录，近亿的单词量，所以超出yarn的虚拟内存资源分配， yarn资源不足 该错误是YARN的虚拟内存计算方式导致，上例中用户程序申请的内存为1Gb，YARN根据此值乘以一个比例（默认为2.1）得出申请的虚拟内存的值，当YARN计算的用户程序所需虚拟内存值大于计算出来的值时，就会报出以上
CDH Hive执行聚合函数报错return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask脱坑指南
妖精小狗的博客
08-23 7702
一、组件版本 我用的是CDH5.16.1，Hive版本是1.1.0 二、出现问题 1. 问题一 普通查询很正常但是聚合查询就会报错，然后去查找日志发现在本地没有\tmp\logs文件，这个文件是用来存放Yarn的log日志的地方，之后查看CDH文档发现yarn的job日志同一存放在【http://XXXX（自己的节点IP）:19888/jobhistory】中找到报错的job。打开发现报错...
解决Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
weixin_47303949的博客
08-12 4617
hive在运行hql语句时报如下错误： Error: Error while processing statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask (state=08S01,code=2) 原因：Hadoop内存不足导致 解决方法： 在Hadoop的配置文件Hadoop-env.sh中添加如下配置： export HADOOP_CLIENT_OPTS="-
Hive报错FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
qq_45124566的博客
07-05 2032
Hive报错FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
Hive报错:FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
热门推荐
荒野雄兵的专栏
11-28 2万+
Hive报错，主要错误信息： java.sql.SQLException: Error while processing statement: FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask 原因分析： Hive sql语句过长，并在sql中调用自己本...
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask MapReduce Jobs Launched:
最新发布
06-18
遇到"FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask"错误通常意味着在Hadoop或Hive MapReduce作业执行过程中出现了问题，导致任务没有成功完成并返回了非零的错误代码2。这可能由多种原因引起，包括但不限于： 1. 输入/输出数据错误：可能是数据格式不正确、文件路径问题或者权限不足。 2. 内存或资源限制：如果任务所需的内存超出分配或磁盘空间不足，可能导致失败。 3. 程序逻辑错误：Map或Reduce函数中的代码错误可能会导致任务崩溃。 4. Hive元数据问题：如果元数据与实际数据不匹配，如表结构或分区信息错误。 5. 网络问题：作业间的通信问题，如网络中断或节点间的通信瓶颈。 为了解决这个问题，你可以尝试以下步骤： - 查看Hive的日志文件（通常是hadoop logs目录下）以获取更详细的错误信息。 - 检查作业的配置，包括内存分配、map/reduce的数量和大小等。 - 使用`DESCRIBE FORMATTED <table>`检查表结构是否正确，并确认输入和期望输出的数据类型是否匹配。 - 如果是分布式环境，确保所有Hadoop集群节点都在运行并且状态良好。 - 清理并重新执行任务，有时候简单的重启服务可能会解决问题。
评论 1
添加红包

请填写红包祝福语或标题

个

红包个数最小为10个

元

红包金额最低5元

当前余额3.43元 前往充值 >
需支付：10.00元
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包

打赏作者

北慕辰

你的鼓励是小北最大的动力

¥1 ¥2 ¥4 ¥6 ¥10 ¥20
扫码支付：¥1
获取中
扫码支付

您的余额不足，请更换扫码支付或充值

打赏作者

实付元
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明：

1.余额是钱包充值的虚拟货币，按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载，可以购买VIP、付费专栏及课程。

余额充值