在使用sqoop从mysql迁移数据到hive时遇到问题，主要是因为hive表和mysql表结构差异，具体表现为mysql表中有自增id，而hive表中没有。迁移过程中遇到Job Counters显示的任务失败和kill情况，最终导致Export job失败。为解决此问题，需要考虑如何处理mysql中的保留字和自增id，以确保数据顺利迁移。
INFO mapreduce.Job: Counters: 12
Job Counters
Failed map tasks=1
Killed map tasks=3
Launched map tasks=4
Rack-local map tasks=4
Total time spent by all maps in occupied slots (ms)=32585
Total time spent by all reduces in occupied slots (ms)=0
Total time spent by all map tasks (ms)=32585
Total vcore-seconds taken by all map tasks=32585
Total megabyte-seconds taken by all map tasks=33367040
Map-Reduce Framework
CPU time spent (ms)=0
Physical memory (bytes) snapshot=0
Virtual memory (bytes) snapshot=0
WARN mapreduce.Counters: Group FileSystemCounters is deprecated. Use org.apache.hadoop.mapreduce.FileSystemCounter instead
INFO mapreduce.ExportJobBase: Transferred 0 bytes in 21.6711 seconds (0 bytes/sec)
INFO mapreduce.ExportJobBase: Exported 0 records.
ERROR tool.ExportTool: Error during export:
Export job failed!
at org.apache.sqoop.mapreduce.ExportJobBase.runExport(ExportJobBase.
at org.apache.sqoop.manager.SqlManager.exportTable(SqlManager.
at org.apache.sqoop.tool.ExportTool.exportTable(ExportTool.
at org.apache.sqoop.tool.ExportTool.run(ExportTool.
at org.apache.sqoop.Sqoop.run(Sqoop.
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.
at org.apache.sqoop.Sqoop.runTool(Sqoop.
at org.apache.sqoop.Sqoop.runTool(Sqoop.
at org.apache.sqoop.Sqoop.main(Sqoop.
问题解决：
主要是由于hive表和mysql表中的结构不一样，在mysql中多了一个自增id。
扫一扫
3806
被折叠的 条评论
为什么被折叠?
到【灌水乐园】发言
