sqoop将数据从hive到mysql报错ERROR mapreduce.ExportJobBase: Export job failed!

最新推荐文章于 2024-01-04 13:27:31 发布

H_sir_Y

最新推荐文章于 2024-01-04 13:27:31 发布

阅读量7.4k

点赞数 1

文章标签： hive sqoop

本文链接：https://blog.csdn.net/H_sir_Y/article/details/108000383

版权

博客讲述了在使用sqoop从hive导数据到mysql时遇到的ERROR ExportJobBase: Export job failed!错误。问题源于分隔符设置不当，作为分隔符导致所有字段被塞入第一个，其余为null。解决方案是将分隔符改为01，确保正确对应建表语句中的字段，避免null值问题。

摘要由CSDN通过智能技术生成

报错如下

20/08/14 10:57:43 INFO mapreduce.Job: Job job_1595919753289_4416 failed with state FAILED due to: Task failed task_1595919753289_4416_m_000000
Job failed as tasks failed. failedMaps:1 failedReduces:0 killedMaps:0 killedReduces: 0

20/08/14 10:57:43 INFO mapreduce.Job: Counters: 8
        Job Counters
                Failed map tasks=1
                Launched map tasks=1
                Rack-local map tasks=1
                Total time spent by all maps in occupied slots (ms)=4340
                Total time spent by all reduces in occupied slots (ms)=0
                Total time spent by all map tasks (ms)=4340
                Total vcore-milliseconds taken by all map tasks=4340
                Total megabyte-milliseconds taken by all map tasks=4444160
20/08/14 10:57:43 WARN mapreduce.Counters: Group FileSystemCounters is deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead
20/08/14 10:57:43 INFO mapreduce.ExportJobBase: Transferred 0 bytes in 16.7826 seconds (0 bytes/sec)
20/08/14 10:57:43 WARN mapreduce.Counters: Group org.apache.hadoop.mapred.Task$Counter is deprecated. Use org.apache.hadoop.mapreduce.TaskCounter instead
20/08/14 10:57:43 INFO mapreduce.ExportJobBase: Exported 0 records.
20/08/14 10:57:43 ERROR mapreduce.ExportJobBase: Export job failed!
20/08/14 10:57:43 ERROR tool.ExportTool: Error during export:
Export job failed!
        at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.java:444)
        at org.apache.sqoop.manager.MySQLManager.upsertTable(MySQLManager.java:142)
        at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.java:86)
        at org.apache.sqoop.tool.ExportTool.run(ExportTool.java:112)
        at org.apache.sqoop.Sqoop.run(Sqoop.java:146)
        at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
        at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:182)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:233)
        at org.apache.sqoop.Sqoop.runTool(Sqoop.java:242)
        at org.apache.sqoop.Sqoop.main(Sqoop.java:251)
shell命令执行错误

查看hive执行日志发现如下报错

2020-08-14 10:57:42,004 ERROR [IPC Server handler 0 on 35343] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Task: attempt_1595919753289_4416_m_000000_0 - exited : java.io.IOException: Can't export data, please check failed map task logs
	at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:121)
	at org.apache.sqoop.mapreduce.TextExportMapper.map(TextExportMapper.java:38)
	at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:146)
	at org.apache.sqoop.mapreduce.AutoProgressMapper.run(Au