hive insert overwrite directory 报错

hive insert overwrite directory 报错：

2017-06-07T16:56:11,766 ERROR [main]: exec.Task (:()) - Failed with exception Unable to move source hdfs://testhadoop/tmp/t1/.hive-staging_hive_2017-06-07_16-56-05_916_7465107102137731594-1/-ext-10000 to destination /tmp/t1
org.apache.hadoop.hive.ql.metadata.HiveException: Unable to move source hdfs://testhadoop/tmp/t1/.hive-staging_hive_2017-06-07_16-56-05_916_7465107102137731594-1/-ext-10000 to destination /tmp/t1
        at org.apache.hadoop.hive.ql.exec.MoveTask.moveFile(MoveTask.java:103)
        at org.apache.hadoop.hive.ql.exec.MoveTask.execute(MoveTask.java:254)
        at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:197)
        at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:100)
        at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1858)
        at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1562)
        at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1313)
        at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1084)
        at org.apache.hadoop.hive.ql.Driver.run(Driver.java:1072)
        at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:232)
        at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:183)
        at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:399)
        at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:776)
        at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:714)
        at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:641)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:497)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:136)

Caused by: org.apache.hadoop.hive.ql.metadata.HiveException: java.io.IOException: rename for src path: hdfs://testhadoop/tmp/t1/.hive-staging_hive_2017-06-07_16-56-05_916_7465107102137731594-1/-ext-10000/000000_0 to dest path:/tmp/t1/000000_0 returned false
        at org.apache.hadoop.hive.ql.metadata.Hive.moveFile(Hive.java:2942)
        at org.apache.hadoop.hive.ql.exec.MoveTask.moveFileInDfs(MoveTask.java:118)
        at org.apache.hadoop.hive.ql.exec.MoveTask.moveFile(MoveTask.java:96)
        ... 20 more
Caused by: java.io.IOException: rename for src path: hdfs://testhadoop/tmp/t1/.hive-staging_hive_2017-06-07_16-56-05_916_7465107102137731594-1/-ext-10000/000000_0 to dest path:/tmp/t1/000000_0 returned false
        at org.apache.hadoop.hive.ql.metadata.Hive$3.call(Hive.java:2922)
        at org.apache.hadoop.hive.ql.metadata.Hive$3.call(Hive.java:2911)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)

原因：
hive2.1.0bug，升级到hive2.1.1
报错原因是insert overwrite directory ，如果原来的directory已经有数据，mv会报错。详见HIVE-13997，地址https://issues.apache.org/jira/browse/HIVE-13997

bug修复思路： 在mv结果数据到目标目录的之前，先删除目标目录的数据。这个策略与hive1.2.1版本不同。

打patch，https://issues.apache.org/jira/browse/HIVE-13997 下载HIVE-13997.2.patch，然后将path放到hive-2.1.0-src，执行patch -p1 < HIVE-13997.2.patch ; 然后重新编译ql模块，得到hive-exec-2.1.0.jar