dolphinscheduler版本:apache-dolphinscheduler-2.0.3-bin
sqoop版本:1.4.7
在任务调度中不断的报错说没有驱动,但是我查看了 dolphinscheduler 的 lib 和 sqoop 的 lib 目录,发现都有 mysql 的驱动,而且版本符合安装的版本。
后来发现是 sqoop 的参数没有配置全
需要在 dolphinscheduler 的配置页面添加如下参数:
–driver com.mysql.jdbc.Driver
2022-03-18 00:19:21,429 ERROR sqoop.Sqoop: Got exception running Sqoop: java.lang.RuntimeException: Could not load db driver class: com.mysql.jdbc.Driver
java.lang.RuntimeException: Could not load db driver class: com.mysql.jdbc.Driver
at org.apache.sqoop.manager.SqlManager.makeConnection(SqlManager.java:875)
at org.apache.sqoop.manager.GenericJdbcManager.getConnection(GenericJdbcManager.java:59)
at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:763)
at org.apache.sqoop.manager.SqlManager.execute(SqlManager.java:786)
at org.apache.sqoop.manager.SqlManager.getColumnInfoForRawQuery(SqlManager.java:289)
at org.apache.sqoop.manager.SqlManager.getColumnTypesForRawQuery(SqlManager.java:260)
at org.apache.sqoop.manager.SqlManager.getColumnTypesForQuery(SqlManager.java:253)
at org.apache.sqoop.manager.ConnManager.getColumnTypes(ConnManager.java:336)
at org.apache.sqoop.orm.ClassWriter.getColumnTypes(ClassWriter.java:1872)
at org.apache.sqoop.orm.ClassWriter.generate(ClassWriter.java:1671)
at org.apache.sqoop.tool.CodeGenTool.generateORM(CodeGenTool.java:106)
at org.apache.sqoop.tool.ImportTool.importTable(ImportTool.java:501)
at org.apache.sqoop.tool.ImportTool.run(ImportTool.java:628)
at org.apache.sqoop.Sqoop.run(Sqoop.java:147)
at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:76)
at org.apache.sqoop.Sqoop.runSqoop(Sqoop.java:183)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:234)
at org.apache.sqoop.Sqoop.runTool(Sqoop.java:243)
at org.apache.sqoop.Sqoop.main(Sqoop.java:252)
找不到 sqoop 命令:
- SQOOP_HOME路径,环境变量要配置,dolphinscheduler 的 /opt/module/ds/conf/env/dolphinscheduler_env.sh 中也要配置
- 每个服务节点都要有 sqoop
[INFO] 2022-03-18 00:50:14.559 TaskLogLogger-class org.apache.dolphinscheduler.plugin.task.sqoop.SqoopTask:[66] - -> welcome to use bigdata scheduling system...
/tmp/dolphinscheduler/exec/process/4629141225184/4630484344672_8/18/35/18_35.command:行5: sqoop: 未找到命令
[INFO] 2022-03-18 00:50:14.560 TaskLogLogger-class org.apache.dolphinscheduler.plugin.task.sqoop.SqoopTask:[60] - FINALIZE_SESSION