1、出现报错:org.apache.hadoop.hive.ql.exec.MoveTask. Unable to move source
具体报错如下:
FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.MoveTask. Unable to move source hdfs://sd-140-181:8020/user/sd/dwd/my/gmlog/ForgeLog/dt=20221024/.hive-staging_hive_2022-10-27_10-01-48_465_4283255161601798835-1/-ext-10000 to destination hdfs://sd-140-181:8020/user/sd/dwd/my/gmlog/ForgeLog/dt=20221024
MapReduce Jobs Launched:
Stage-Stage-1: Map: 1 Cumulative CPU: 31.65 sec HDFS Read: 19534635 HDFS Write: 7182152 HDFS EC Read: 0 SUCCESS
Total MapReduce CPU Time Spent: 31 seconds 650 msec
20221024sd_dw_my_dwd.gmlog_forgelog_quality数据装载失败
2、解决:根据资料,是元数据问题,需要修复下:
msck repair table sd_dw_my_dwd.gmlog_forgelog_quality;
这里是因为我在跑数前,把hdfs上,dwd文件夹删除了,导致元数据还是存在,修复完成后即正常
参考:
https://mp.weixin.qq.com/s?src=11×tamp=1666838210&ver=4129&signature=cvnMMqH5FN4Y86gL33b0q32N6FRA5S6RqnUUsqj3r-JmGqMzfe0RytK-zZaWjILZcGAHEcHtRt1OXFyQLVpG6D8uMrYOir9ka85LR1oqbcMrRHsWl3xSLYrrl3THhsme&new=1