Task with the most failures(4):
-----
Task ID:
task_1555665274604_56333_r_000000
URL:
http://0.0.0.0:8088/taskdetails.jsp?jobid=job_1555665274604_56333&tipid=task_1555665274604_56333_r_000000
-----
Diagnostic Messages for this Task:
Error: org.apache.hadoop.mapreduce.task.reduce.Shuffle$ShuffleError: error in shuffle in OnDiskMerger - Thread to merge on-disk map-outputs
at org.apache.hadoop.mapreduce.task.reduce.Shuffle.run(Shuffle.java:166)
at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:376)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:164)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1707)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.io.IOException: Rec# 12307: Negative value-length: -1
at org.apache.hadoop.mapred.IFile$Reader.positionToNextRecord(IFile.java:412)
at org.apache.hadoop.mapred.IFile$Reader.nextRawKey(IFile.java:420)
at org.apache.hadoop.mapred.Merger$Segment.nextRawKey(Merger.java:337)
at org.apache.hadoop.mapred.Merger$MergeQueue.adjustPriorityQueue(Merger.java:519)
at org.apache.hadoop.mapred.Merger$MergeQueue.next(Merger.java:547)
at org.apache.hadoop.mapred.Merger.writeFile(Merger.java:209)
at org.apache.hadoop.mapreduce.task.reduce.MergeManagerImpl$OnDiskMerger.merge(MergeManagerImpl.java:565)
at org.apache.hadoop.mapreduce.task.reduce.MergeThread.run(MergeThread.java:94)
FAILED: Execution Error, return code 2 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
MapReduce Jobs Launched:
Stage-Stage-1: Map: 5541 Reduce: 100 Cumulative CPU: 1.166330597E7 sec HDFS Read: 18561912740619 HDFS Write: 0 FAIL
Total MapReduce CPU Time Spent: 134 days 23 hours 48 minutes 25 seconds 970 msec
这个错误我猜测是因为set mapreduce.reduce.shuffle.input.buffer.percent=0.3;设的太大了,导致往硬盘写入大量数据,占满了磁盘,可以将值设的小一点