用sqoop定时增量读取mysql数据并导入hdfs中,通常在linux平台启动hdf后,然后输入
sqoop job --show myjob
通常会显示正确的信息,但是有时候会报错,信息如下
[root@hadoop bin]# ./sqoop job --show myjob
19/06/02 11:48:21 INFO sqoop.Sqoop: Running Sqoop version: 1.4.6-cdh5.7.0
19/06/02 11:48:22 ERROR hsqldb.HsqldbJobStorage: Cannot restore job: myjob
19/06/02 11:48:22 ERROR hsqldb.HsqldbJobStorage: (No such job)
19/06/02 11:48:22 ERROR tool.JobTool: I/O error performing job operation: java.io.IOException: Cannot restore missing job myjob
at org.apache.sqoop.metastore.hsqldb.HsqldbJobStorage.read(HsqldbJobStorage.java:256)
at org.apache.sqoop.tool.JobTool.showJob(JobTool.java:232)
at org.apache.sqoop.tool.JobTool.run(JobTool.java:287)
at org.apache.sqoop.Sqoop.run(Sqoop.java:143)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:179)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:218)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:227)
at org.apache.sqoop.Sqoop.main(Sqoop.java:236)
报错说No such job,也许是以前创建job已经停止运行了,所以我输入
sqoop job --create myjob
-- import --connect jdbc:mysql://192.168.56.1:3306/fangsou
--username root --password-file file:///usr/local/src/sqoop-1.4.6-cdh5.7.0/.mysql.password --table ershoufang -m 1
--target-dir /home/dyw/hdfs/data/ershoufang
--check-column "id"
--incremental "append"
--last-value "231"
--verbose
重新创建job