sqoop报错ERROR tool.ImportTool: Encountered IOException running import job: java.io.IOException: Query

最新推荐文章于 2023-05-09 17:32:27 发布

zyj_369

最新推荐文章于 2023-05-09 17:32:27 发布

阅读量2.4k

点赞数

分类专栏： sqoop 文章标签： sqoop

本文链接：https://blog.csdn.net/weixin_46122692/article/details/112478874

版权

sqoop 专栏收录该内容

1 篇文章 0 订阅

订阅专栏

一、详细报错

ERROR tool.ImportTool: Encountered IOException running import job: java.io.IOException: Query [select id,login_name from user_info where id >= 10 and id <= 30] must contain '$CONDITIONS' in WHERE clause.
	at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:300)
	at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1833)
	at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1645)
	at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:107)
	at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:478)
	at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:605)
	at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
	at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
	at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
	at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
	at org.apache.sqoop.Sqoop.main(Sqoop.java:236)

二、报错原因

我之前的配置是这么写的，$CONDITIONS相当于是占位符，sqoop通过他来获取分片信息，应该由一个map读取多少条数据。

bin/sqoop import \
--connect jdbc:mysql://hadoop101:3306/gmall \
--username root \
--password 000000 \
--query 'select id,login_name from user_info where id >= 10 and id <= 30' \
--target-dir /test \
--delete-target-dir \
--num-mappers 2 \
--fields-terminated-by '\t' \
--split-by id

三、解决

将配置修改成下方的即可。

bin/sqoop import \
--connect jdbc:mysql://hadoop101:3306/gmall \
--username root \
--password 000000 \
--query 'select id,login_name from user_info where id >= 10 and id <= 30 and $CONDITIONS' \
--target-dir /test \
--delete-target-dir \
--num-mappers 2 \
--fields-terminated-by '\t' \
--split-by id