具体报错如下:
2023-11-28 10:42:13,844 INFO hive.HiveImport: Hive Session ID = b3193c5a-673b-4bff-832e-afb887559e8e
2023-11-28 10:42:15,162 INFO hive.HiveImport: FAILED: SemanticException [Error 10035]: Column repeated in partitioning columns
2023-11-28 10:42:15,629 ERROR tool.ImportTool: Encountered IOException running import job: java.io.IOException: Hive exited with status 51
at org.apache.sqoop.hive.HiveImport.executeExternalHiveScript(HiveImport.java:389)
at org.apache.sqoop.hive.HiveImport.executeScript(HiveImport.java:339)
at org.apache.sqoop.hive.HiveImport.importTable(HiveImport.java:240)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:514)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
原因:
因为我在用sqoop导出到hive时添加了一个分区的字段,而导出的表中刚好存在这个字段,和分区字段冲突,故报此错误
解决方法:更改分区字段名称
--hive-partition-key ptr
注意:如果分区字段是有含义的数据类型也会报此错误