一、详细报错
ERROR tool.ImportTool: Encountered IOException running import job: java.io.IOException: Query [select id,login_name from user_info where id >= 10 and id <= 30] must contain '$CONDITIONS' in WHERE clause.
at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:300)
at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1833)
at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1645)
at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:107)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:478)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
二、报错原因
我之前的配置是这么写的,$CONDITIONS相当于是占位符,sqoop通过他来获取分片信息,应该由一个map读取多少条数据。
bin/sqoop import \
--connect jdbc:mysql://hadoop101:3306/gmall \
--username root \
--password 000000 \
--query 'select id,login_name from user_info where id >= 10 and id <= 30' \
--target-dir /test \
--delete-target-dir \
--num-mappers 2 \
--fields-terminated-by '\t' \
--split-by id
三、解决
将配置修改成下方的即可。
bin/sqoop import \
--connect jdbc:mysql://hadoop101:3306/gmall \
--username root \
--password 000000 \
--query 'select id,login_name from user_info where id >= 10 and id <= 30 and $CONDITIONS' \
--target-dir /test \
--delete-target-dir \
--num-mappers 2 \
--fields-terminated-by '\t' \
--split-by id