说明:使用本机的mysql数据库,将mysql数据库的表的数据 导入 hdfs 中
配置好同步文件后出现以下错误:
2021-05-12 22:13:22.968 [job-0] WARN DBUtil - test connection of [jdbc:mysql://10.9.68.211:3306/testdb] failed, for Code:[DBUtilErrorCode-10], Description:[连接数据库失败. 请检查您的 账号、密码、数据库名称、IP、Port或者向 DBA 寻求帮助(注意网络环境).]. - 具体错误信息为:java.sql.SQLException: null, message from server: "Host 'DESKTOP-5J5BBG5' is not allowed to connect to this MySQL server". 2021-05-12 22:13:22.972 [job-0] ERROR RetryUtil - Exception when calling callable, 异常Msg:DataX无法连接对应的数据库,可能原因是:1) 配置的ip/port/database/jdbc错误,无法连接。2) 配置的username/password错误,鉴权失败。请和DBA确认该数据库的连接信息是否正确。 java.lang.Exception: DataX无法连接对应的数据库,可能原因是:1) 配置的ip/port/database/jdbc错误,无法连接。2) 配置的username/password错误,鉴权失败。请和DBA确认该数据库的连接信息是否正确。 at com.alibaba.datax.plugin.rdbms.util.DBUtil$2.call(DBUtil.java:71) ~[plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na] at com.alibaba.datax.plugin.rdbms.util.DBUtil$2.call(DBUtil.java:51) ~[plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na] at com.alibaba.datax.common.util.RetryUtil$Retry.call(RetryUtil.java:164) ~[datax-common-0.0.1-SNAPSHOT.jar:na] at com.alibaba.datax.common.util.RetryUtil$Retry.doRetry(RetryUtil.java:111) ~[datax-common-0.0.1-SNAPSHOT.jar:na] at com.alibaba.datax.common.util.RetryUtil.executeWithRetry(RetryUtil.java:30) [datax-common-0.0.1-SNAPSHOT.jar:na] at com.alibaba.datax.plugin.rdbms.util.DBUtil.chooseJdbcUrl(DBUtil.java:51) [plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na] at com.alibaba.datax.plugin.rdbms.reader.util.OriginalConfPretreatmentUtil.dealJdbcAndTable(OriginalConfPretreatmentUtil.java:92) [plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na] at com.alibaba.datax.plugin.rdbms.reader.util.OriginalConfPretreatmentUtil.simplifyConf(OriginalConfPretreatmentUtil.java:59) [plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na] at com.alibaba.datax.plugin.rdbms.reader.util.OriginalConfPretreatmentUtil.doPretreatment(OriginalConfPretreatmentUtil.java:33) [plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na] at com.alibaba.datax.plugin.rdbms.reader.CommonRdbmsReader$Job.init(CommonRdbmsReader.java:55) [plugin-rdbms-util-0.0.1-SNAPSHOT.jar:na] at com.alibaba.datax.plugin.reader.mysqlreader.MysqlReader$Job.init(MysqlReader.java:37) [mysqlreader-0.0.1-SNAPSHOT.jar:na] at com.alibaba.datax.core.job.JobContainer.initJobReader(JobContainer.java:673) [datax-core-0.0.1-SNAPSHOT.jar:na] at com.alibaba.datax.core.job.JobContainer.init(JobContainer.java:303) [datax-core-0.0.1-SNAPSHOT.jar:na] at com.alibaba.datax.core.job.JobContainer.start(JobContainer.java:113) [datax-core-0.0.1-SNAPSHOT.jar:na] at com.alibaba.datax.core.Engine.start(Engine.java:92) [datax-core-0.0.1-SNAPSHOT.jar:na] at com.alibaba.datax.core.Engine.entry(Engine.java:171) [datax-core-0.0.1-SNAPSHOT.jar:na] at com.alibaba.datax.core.Engine.main(Engine.java:204) [datax-core-0.0.1-SNAPSHOT.jar:na] 2021-05-12 22:13:23.973 [job-0] ERROR RetryUtil - Exception when calling callable, 即将尝试执行第1次重试.本次重试计划等待[1000]ms,实际等待[1001]ms, 异常Msg:[DataX无法连接对应的数据库,可能原因是:1) 配置的ip/port/database/jdbc错误,无法连接。2) 配置的username/password错误,鉴权失败。请和DBA确认该数据库的连接信息是否正确。] 2021-05-12 22:13:23.978 [job-0] WARN DBUtil - test connection of [jdbc:mysql://10.9.68.211:3306/testdb] failed, for Code:[DBUtilErrorCode-10], Description:[连接数据库失败. 请检查您的 账号、密码、数据库名称、IP、Port或者向 DBA 寻求帮助(注意网络环境).]. - 具体错误信息为:java.sql.SQLException: null, message from server: "Host 'DESKTOP-5J5BBG5' is not allowed to connect to this MySQL server".
解决办法:
本地数据库的权限问题,只要对本地的mysql数据库放开权限就可以了
执行以下命令即可
grant all privileges on *.* to 'root'@'%' IDENTIFIED BY '123456';
FLUSH PRIVILEGES;
同步文件:
{ "job": { "setting": { "speed": { "channel": 2 } }, "content": [{ "reader": { "name": "mysqlreader", "parameter": { "username": "root", "password": "123456", "column": [ "category_code", "category_name", "category_parent_id", "category_status", "category_utime" ], "splitPk": "category_code", "where": "category_utime >= '${start_time}' and category_utime < '${end_time}'", "connection": [{ "table": [ "category" ], "jdbcUrl": [ "jdbc:mysql://10.9.68.211:3306/testdb" ] }] } }, "writer": { "name": "hdfswriter", "parameter": { "defaultFS": "hdfs://master:9000", "fileType": "text", "path": "/datax/mysql2hdfs/TestAppend", "fileName": "m2h01", "column": [{ "name": "category_code", "type": "VARCHAR" }, { "name": "category_name", "type": "VARCHAR" }, { "name": "category_parent_id", "type": "VARCHAR" }, { "name": "category_status", "type": "tinyint" }, { "name": "category_utime", "type": "bigint" } ], "writeMode": "append", "fieldDelimiter": "\t" } } }] } }