背景
datax没有kerberos认证,原因是阿里觉得kerberos太坑了(我觉得也是),奈何公司用的kerberos的现状是在下一人之力无法改变的。
可怜果子无法改变公司,只能去尝试验证通过kerberos。
环境
希望大家看此文时候已经将以下环境准备好
- datax已经安装好。
- datax的opentsdbreader已经安装到datax/plugin/reader下
- opentsdb已经部署好,且已经启动,能够成功访问页面
问题发现
安装好datax以及将 .json文件写好之后,就发现了如下异常
这是在下的json文件
{
"job": {
"setting": {
"speed": {
"channel": 3
},
"errorLimit": {
"record": 0,
"percentage": 0.02
}
},
"content": [
{
"reader": {
"name": "opentsdbreader",
"parameter": {
"endpoint": "http://cdh02:4242",
"column": [
"sys.cpu.nice"
],
"beginDateTime": "2017-01-01 00:00:00",
"endDateTime": "2017-12-01 00:00:00"
}
},
"writer": {
"name": "streamwriter",
"parameter": {
"encoding": "UTF-8",
"print": true
}
}
}
]
}
}
一下为执行时所出现的异常
2020-04-08 10:41:11.058 [job-0] INFO JobContainer - DataX Reader.Job [opentsdbreader] splits to [8016] tasks.
2020-04-08 10:41:11.059 [job-0] INFO JobContainer - DataX Writer.Job [streamwriter] splits to [8016] tasks.
2020-04-08 10:41:11.457 [job-0] INFO JobContainer - jobContainer starts to do schedule ...
2020-04-08 10:44:22.342 [job-0] INFO JobContainer - Scheduler starts [1] taskGroups.
2020-04-08 10:44:22.344 [job-0] INFO JobContainer - Running by standalone Mode.
2020-04-08 10:44:22.589 [taskGroup-0] INFO TaskGroupContainer - taskGroupId=[0] start [3] channels for [8016] tasks.
2020-04-08 10:44:22.621 [taskGroup-0] INFO Channel - Channel set byte_speed_limit to -1, No bps activated.
2020-04-08 10:44:22.621 [taskGroup-0] INFO Channel - Channel set record_speed_limit to -1, No tps activated.
2020-04-08 10:44:22.631 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[2311] attemptCount[1] is started
2020-04-08 10:44:22.632 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[3009] attemptCount[1] is started
2020-04-08 10:44:22.633 [taskGroup-0] INFO TaskGroupContainer - taskGroup[0] taskId[1477] attemptCount[1] is started
2020-04-08 10:44:22.633 [0-0-2311-reader] INFO OpenTSDBReader$Task - getPluginJobConf: {"internal":{"beginDateTime":1491519600000,"column":["sys.cpu.nice"],"endDateTime":1491523199999,"endpoint":"http://cd02:4242"},"keys":["endpoint","column[0]","beginDateTime","endDateTime"],"secretKeyPathSet":[]}
2020-04-08 10:44:22.634 [0-0-3009-reader] INFO OpenTSDBReader$Task - getPluginJobConf: {"internal":{"beginDateTime":1494032400000,"column":["sys.cpu.nice"],"endDateTime":1494035999999,"endpoint":"http://cd02:4242"},"keys":["endpoint","column[0]","beginDateTime","endDateTime"],"secretKeyPathSet":[]}
2020-04-08 10:44:22.636 [0-0-1477-reader] INFO OpenTSDBReader$Task - getPluginJobConf: {"internal":{"beginDateTime":1488517200000,"column":["sys.cpu.nice"],"endDateTime":1488520799999,"endpoint":"http://cd02:4242"},"keys":["endpoint","column[0]","beginDateTime","endDateTime"],"secretKeyPathSet":[]}
2020-04-08 10:44:23.118 [0-0-2311-reader] ERROR ReaderRunner - Reader runner Received Exceptions:
com.alibaba.datax.common.exception.DataXException: Code:[OpenTSDBReader-01], Description:[值非法]. - 获取或发送数据点的过程中出错! - java.lang.RuntimeException: Cannot init OpenTSDB connection!
at com.alibaba.datax.plugin.reader.conn.OpenTSDBDump.getTSDB(OpenTSDBDump.java:41)
at com.alibaba.datax.plugin.reader.conn.OpenTSDBDump.dump(OpenTSDBDump.java:26)
at com.alibaba.datax.plugin.reader.conn.OpenTSDBConnection.sendDPs(OpenTSDBConnection.java:48)
at com.alibaba.datax.plugin.reader.opentsdbreader.OpenTSDBReader$Task.startRead(OpenTSDBReader.java:191)
at com.alibaba.datax.core.taskgroup.runner.ReaderRunner.run(ReaderRunner.java:57)
at java.lang.Thread.run(Thread.java:748)
at com.alibaba.datax.common.exception.DataXException.asDataXException(DataXException.java:33) ~[datax-common-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.reader.opentsdbreader.OpenTSDBReader$Task.startRead(OpenTSDBReader.java:194) ~[opentsdbreader-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.core.taskgroup.runner.ReaderRunner.run(ReaderRunner.java:57) ~[datax-core-0.0.1-SNAPSHOT.jar:na]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_212]
Caused by: java.lang.RuntimeException: Cannot init OpenTSDB connection!
at com.alibaba.datax.plugin.reader.conn.OpenTSDBDump.getTSDB(OpenTSDBDump.java:41) ~[opentsdbreader-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.reader.conn.OpenTSDBDump.dump(OpenTSDBDump.java:26) ~[opentsdbreader-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.reader.conn.OpenTSDBConnection.sendDPs(OpenTSDBConnection.java:48) ~[opentsdbreader-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.reader.opentsdbreader.OpenTSDBReader$Task.startRead(OpenTSDBReader.java:191) ~[opentsdbreader-0.0.1-SNAPSHOT.jar:na]
... 2 common frames omitted
2020-04-08 10:44:23.119 [0-0-1477-reader] ERROR ReaderRunner - Reader runner Received Exceptions:
com.alibaba.datax.common.exception.DataXException: Code:[OpenTSDBReader-01], Description:[值非法]. - 获取或发送数据点的过程中出错! - java.lang.RuntimeException: Cannot init OpenTSDB connection!
at com.alibaba.datax.plugin.reader.conn.OpenTSDBDump.getTSDB(OpenTSDBDump.java:41)
at com.alibaba.datax.plugin.reader.conn.OpenTSDBDump.dump(OpenTSDBDump.java:26)
at com.alibaba.datax.plugin.reader.conn.OpenTSDBConnection.sendDPs(OpenTSDBConnection.java:48)
at com.alibaba.datax.plugin.reader.opentsdbreader.OpenTSDBReader$Task.startRead(OpenTSDBReader.java:191)
at com.alibaba.datax.core.taskgroup.runner.ReaderRunner.run(ReaderRunner.java:57)
at java.lang.Thread.run(Thread.java:748)
at com.alibaba.datax.common.exception.DataXException.asDataXException(DataXException.java:33) ~[datax-common-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.reader.opentsdbreader.OpenTSDBReader$Task.startRead(OpenTSDBReader.java:194) ~[opentsdbreader-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.core.taskgroup.runner.ReaderRunner.run(ReaderRunner.java:57) ~[datax-core-0.0.1-SNAPSHOT.jar:na]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_212]
Caused by: java.lang.RuntimeException: Cannot init OpenTSDB connection!
at com.alibaba.datax.plugin.reader.conn.OpenTSDBDump.getTSDB(OpenTSDBDump.java:41) ~[opentsdbreader-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.reader.conn.OpenTSDBDump.dump(OpenTSDBDump.java:26) ~[opentsdbreader-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.reader.conn.OpenTSDBConnection.sendDPs(OpenTSDBConnection.java:48) ~[opentsdbreader-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.reader.opentsdbreader.OpenTSDBReader$Task.startRead(OpenTSDBReader.java:191) ~[opentsdbreader-0.0.1-SNAPSHOT.jar:na]
... 2 common frames omitted
2020-04-08 10:44:23.120 [0-0-3009-reader] ERROR ReaderRunner - Reader runner Received Exceptions:
com.alibaba.datax.common.exception.DataXException: Code:[OpenTSDBReader-01], Description:[值非法]. - 获取或发送数据点的过程中出错! - java.lang.RuntimeException: Cannot init OpenTSDB connection!
at com.alibaba.datax.plugin.reader.conn.OpenTSDBDump.getTSDB(OpenTSDBDump.java:41)
at com.alibaba.datax.plugin.reader.conn.OpenTSDBDump.dump(OpenTSDBDump.java:26)
at com.alibaba.datax.plugin.reader.conn.OpenTSDBConnection.sendDPs(OpenTSDBConnection.java:48)
at com.alibaba.datax.plugin.reader.opentsdbreader.OpenTSDBReader$Task.startRead(OpenTSDBReader.java:191)
at com.alibaba.datax.core.taskgroup.runner.ReaderRunner.run(ReaderRunner.java:57)
at java.lang.Thread.run(Thread.java:748)
at com.alibaba.datax.common.exception.DataXException.asDataXException(DataXException.java:33) ~[datax-common-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.reader.opentsdbreader.OpenTSDBReader$Task.startRead(OpenTSDBReader.java:194) ~[opentsdbreader-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.core.taskgroup.runner.ReaderRunner.run(ReaderRunner.java:57) ~[datax-core-0.0.1-SNAPSHOT.jar:na]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_212]
Caused by: java.lang.RuntimeException: Cannot init OpenTSDB connection!
at com.alibaba.datax.plugin.reader.conn.OpenTSDBDump.getTSDB(OpenTSDBDump.java:41) ~[opentsdbreader-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.reader.conn.OpenTSDBDump.dump(OpenTSDBDump.java:26) ~[opentsdbreader-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.reader.conn.OpenTSDBConnection.sendDPs(OpenTSDBConnection.java:48) ~[opentsdbreader-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.reader.opentsdbr