HiveException: xx with newer attempt ID 1 is smaller than the file xx with with older attempt ID 0

版本：HDP3.1.4 Hive3.1.0

异常描述：

java.io.IOException: File hdfs://cluster/tmp/hive/.hive-staging_hive_2021-04-09_10-08-47_704_718343819133902574-25365/_tmp.-ext-10002.moved/000316_1 with newer attempt ID 1 is smaller than the file hdfs://cluster/tmp/hive/.hive-staging_hive_2021-04-09_10-08-47_704_718343819133902574-25365/_tmp.-ext-10002.moved/000316_0 with older attempt ID 0)'
13929: 2021-04-09 10:24:43.608 | com.bx.slave.entity.JobTask.runJob(436) | org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException: File hdfs://cluster/tmp/hive/.hive-staging_hive_2021-04-09_10-08-47_704_718343819133902574-25365/_tmp.-ext-10002.moved/000316_1 with newer attempt ID 1 is smaller than the file hdfs://cluster/tmp/hive/.hive-staging_hive_2021-04-09_10-08-47_704_718343819133902574-25365/_tmp.-ext-10002.moved/000316_0 with older attempt ID 0
13930: 2021-04-09 10:24:43.608 | at org.apache.hadoop.hive.ql.exec.FileSinkOperator.jobCloseOp(FileSinkOperator.java:1402)
13931: 2021-04-09 10:24:43.608 | at org.apache.hadoop.hive.ql.exec.Operator.jobClose(Operator.java:798)
13932: 2021-04-09 10:24:43.608 | at org.apache.hadoop.hive.ql.exec.Operator.jobClose(Operator.java:803)
13933: 2021-04-09 10:24:43.608 | at org.apache.hadoop.hive.ql.exec.Operator.jobClose(Operator.java:803)
13934: 2021-04-09 10:24:43.608 | at org.apache.hadoop.hive.ql.exec.tez.TezTask.close(TezTask.java:584)
13935: 2021-04-09 10:24:43.608 | at org.apache.hadoop.hive.ql.exec.tez.TezTask.execute(TezTask.java:321)
13936: 2021-04-09 10:24:43.609 | at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:212)
13937: 2021-04-09 10:24:43.609 | at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:103)
13938: 2021-04-09 10:24:43.609 | at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:2712)
13939: 2021-04-09 10:24:43.609 | at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:2383)
13940: 2021-04-09 10:24:43.609 | at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:2055)
13941: 2021-04-09 10:24:43.609 | at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1753)
13942: 2021-04-09 10:24:43.609 | at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1747)
13943: 2021-04-09 10:24:43.609 | at org.apache.hadoop.hive.ql.reexec.ReExecDriver.run(ReExecDriver.java:157)
13944: 2021-04-09 10:24:43.609 | at org.apache.hive.service.cli.operation.SQLOperation.runQuery(SQLOperation.java:226)
13945: 2021-04-09 10:24:43.609 | at org.apache.hive.service.cli.operation.SQLOperation.access$700(SQLOperation.java:87)
13946: 2021-04-09 10:24:43.609 | at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork$1.run(SQLOperation.java:324)
13947: 2021-04-09 10:24:43.609 | at java.security.AccessController.doPrivileged(Native Method)
13948: 2021-04-09 10:24:43.609 | at javax.security.auth.Subject.doAs(Subject.java:422)
13949: 2021-04-09 10:24:43.609 | at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1730)
13950: 2021-04-09 10:24:43.609 | at org.apache.hive.service.cli.operation.SQLOperation$BackgroundWork.run(SQLOperation.java:342)
13951: 2021-04-09 10:24:43.609 | at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
13952: 2021-04-09 10:24:43.609 | at java.util.concurrent.FutureTask.run(FutureTask.java:266)
13953: 2021-04-09 10:24:43.610 | at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
13954: 2021-04-09 10:24:43.610 | at java.util.concurrent.FutureTask.run(FutureTask.java:266)
13955: 2021-04-09 10:24:43.610 | at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
13956: 2021-04-09 10:24:43.610 | at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
13957: 2021-04-09 10:24:43.610 | at java.lang.Thread.run(Thread.java:748)
13958: 2021-04-09 10:24:43.610 | Caused by: java.io.IOException: File hdfs://cluster/tmp/hive/.hive-staging_hive_2021-04-09_10-08-47_704_718343819133902574-25365/_tmp.-ext-10002.moved/000316_1 with newer attempt ID 1 is smaller than the file hdfs://cluster/tmp/hive/.hive-staging_hive_2021-04-09_10-08-47_704_718343819133902574-25365/_tmp.-ext-10002.moved/000316_0 with older attempt ID 0

重试文件的大小可能和压缩格式和写入顺序有关系，所以在校验id和文件大小的方式不恰当,因此去掉文件大小校验

源码地址：Utilities.java   

compareTempOrDuplicateFiles

PATCH地址：
https://issues.apache.org/jira/browse/HIVE-23354

（可以关闭mr和hive的推测执行，应用patch1）