DataX opentsdb reader 通过kerberos认证

背景

datax没有kerberos认证,原因是阿里觉得kerberos太坑了(我觉得也是),奈何公司用的kerberos的现状是在下一人之力无法改变的。
可怜果子无法改变公司,只能去尝试验证通过kerberos。

环境

希望大家看此文时候已经将以下环境准备好

  • datax已经安装好。
  • datax的opentsdbreader已经安装到datax/plugin/reader下
  • opentsdb已经部署好,且已经启动,能够成功访问页面

问题发现

安装好datax以及将 .json文件写好之后,就发现了如下异常
这是在下的json文件

{
  "job": {
    "setting": {
      "speed": {
        "channel": 3
      },
      "errorLimit": {
        "record": 0,
        "percentage": 0.02
      }
    },
    "content": [
       {
            "reader": {
    "name": "opentsdbreader",
    "parameter": {
        "endpoint": "http://cdh02:4242",
        "column": [
                   "sys.cpu.nice"
        ],
        "beginDateTime": "2017-01-01 00:00:00",
        "endDateTime": "2017-12-01 00:00:00"
    }
},
       "writer": {
          "name": "streamwriter",
          "parameter": {
            "encoding": "UTF-8",
            "print": true
          }
        }
       }
    ]
  }
}

一下为执行时所出现的异常

2020-04-08 10:41:11.058 [job-0] INFO  JobContainer - DataX Reader.Job [opentsdbreader] splits to [8016] tasks.
2020-04-08 10:41:11.059 [job-0] INFO  JobContainer - DataX Writer.Job [streamwriter] splits to [8016] tasks.
2020-04-08 10:41:11.457 [job-0] INFO  JobContainer - jobContainer starts to do schedule ...
2020-04-08 10:44:22.342 [job-0] INFO  JobContainer - Scheduler starts [1] taskGroups.
2020-04-08 10:44:22.344 [job-0] INFO  JobContainer - Running by standalone Mode.
2020-04-08 10:44:22.589 [taskGroup-0] INFO  TaskGroupContainer - taskGroupId=[0] start [3] channels for [8016] tasks.
2020-04-08 10:44:22.621 [taskGroup-0] INFO  Channel - Channel set byte_speed_limit to -1, No bps activated.
2020-04-08 10:44:22.621 [taskGroup-0] INFO  Channel - Channel set record_speed_limit to -1, No tps activated.
2020-04-08 10:44:22.631 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] taskId[2311] attemptCount[1] is started
2020-04-08 10:44:22.632 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] taskId[3009] attemptCount[1] is started
2020-04-08 10:44:22.633 [taskGroup-0] INFO  TaskGroupContainer - taskGroup[0] taskId[1477] attemptCount[1] is started
2020-04-08 10:44:22.633 [0-0-2311-reader] INFO  OpenTSDBReader$Task - getPluginJobConf: {"internal":{"beginDateTime":1491519600000,"column":["sys.cpu.nice"],"endDateTime":1491523199999,"endpoint":"http://cd02:4242"},"keys":["endpoint","column[0]","beginDateTime","endDateTime"],"secretKeyPathSet":[]}
2020-04-08 10:44:22.634 [0-0-3009-reader] INFO  OpenTSDBReader$Task - getPluginJobConf: {"internal":{"beginDateTime":1494032400000,"column":["sys.cpu.nice"],"endDateTime":1494035999999,"endpoint":"http://cd02:4242"},"keys":["endpoint","column[0]","beginDateTime","endDateTime"],"secretKeyPathSet":[]}
2020-04-08 10:44:22.636 [0-0-1477-reader] INFO  OpenTSDBReader$Task - getPluginJobConf: {"internal":{"beginDateTime":1488517200000,"column":["sys.cpu.nice"],"endDateTime":1488520799999,"endpoint":"http://cd02:4242"},"keys":["endpoint","column[0]","beginDateTime","endDateTime"],"secretKeyPathSet":[]}
2020-04-08 10:44:23.118 [0-0-2311-reader] ERROR ReaderRunner - Reader runner Received Exceptions:
com.alibaba.datax.common.exception.DataXException: Code:[OpenTSDBReader-01], Description:[值非法].  - 获取或发送数据点的过程中出错! - java.lang.RuntimeException: Cannot init OpenTSDB connection!
	at com.alibaba.datax.plugin.reader.conn.OpenTSDBDump.getTSDB(OpenTSDBDump.java:41)
	at com.alibaba.datax.plugin.reader.conn.OpenTSDBDump.dump(OpenTSDBDump.java:26)
	at com.alibaba.datax.plugin.reader.conn.OpenTSDBConnection.sendDPs(OpenTSDBConnection.java:48)
	at com.alibaba.datax.plugin.reader.opentsdbreader.OpenTSDBReader$Task.startRead(OpenTSDBReader.java:191)
	at com.alibaba.datax.core.taskgroup.runner.ReaderRunner.run(ReaderRunner.java:57)
	at java.lang.Thread.run(Thread.java:748)

	at com.alibaba.datax.common.exception.DataXException.asDataXException(DataXException.java:33) ~[datax-common-0.0.1-SNAPSHOT.jar:na]
	at com.alibaba.datax.plugin.reader.opentsdbreader.OpenTSDBReader$Task.startRead(OpenTSDBReader.java:194) ~[opentsdbreader-0.0.1-SNAPSHOT.jar:na]
	at com.alibaba.datax.core.taskgroup.runner.ReaderRunner.run(ReaderRunner.java:57) ~[datax-core-0.0.1-SNAPSHOT.jar:na]
	at java.lang.Thread.run(Thread.java:748) [na:1.8.0_212]
Caused by: java.lang.RuntimeException: Cannot init OpenTSDB connection!
	at com.alibaba.datax.plugin.reader.conn.OpenTSDBDump.getTSDB(OpenTSDBDump.java:41) ~[opentsdbreader-0.0.1-SNAPSHOT.jar:na]
	at com.alibaba.datax.plugin.reader.conn.OpenTSDBDump.dump(OpenTSDBDump.java:26) ~[opentsdbreader-0.0.1-SNAPSHOT.jar:na]
	at com.alibaba.datax.plugin.reader.conn.OpenTSDBConnection.sendDPs(OpenTSDBConnection.java:48) ~[opentsdbreader-0.0.1-SNAPSHOT.jar:na]
	at com.alibaba.datax.plugin.reader.opentsdbreader.OpenTSDBReader$Task.startRead(OpenTSDBReader.java:191) ~[opentsdbreader-0.0.1-SNAPSHOT.jar:na]
	... 2 common frames omitted
2020-04-08 10:44:23.119 [0-0-1477-reader] ERROR ReaderRunner - Reader runner Received Exceptions:
com.alibaba.datax.common.exception.DataXException: Code:[OpenTSDBReader-01], Description:[值非法].  - 获取或发送数据点的过程中出错! - java.lang.RuntimeException: Cannot init OpenTSDB connection!
	at com.alibaba.datax.plugin.reader.conn.OpenTSDBDump.getTSDB(OpenTSDBDump.java:41)
	at com.alibaba.datax.plugin.reader.conn.OpenTSDBDump.dump(OpenTSDBDump.java:26)
	at com.alibaba.datax.plugin.reader.conn.OpenTSDBConnection.sendDPs(OpenTSDBConnection.java:48)
	at com.alibaba.datax.plugin.reader.opentsdbreader.OpenTSDBReader$Task.startRead(OpenTSDBReader.java:191)
	at com.alibaba.datax.core.taskgroup.runner.ReaderRunner.run(ReaderRunner.java:57)
	at java.lang.Thread.run(Thread.java:748)

	at com.alibaba.datax.common.exception.DataXException.asDataXException(DataXException.java:33) ~[datax-common-0.0.1-SNAPSHOT.jar:na]
	at com.alibaba.datax.plugin.reader.opentsdbreader.OpenTSDBReader$Task.startRead(OpenTSDBReader.java:194) ~[opentsdbreader-0.0.1-SNAPSHOT.jar:na]
	at com.alibaba.datax.core.taskgroup.runner.ReaderRunner.run(ReaderRunner.java:57) ~[datax-core-0.0.1-SNAPSHOT.jar:na]
	at java.lang.Thread.run(Thread.java:748) [na:1.8.0_212]
Caused by: java.lang.RuntimeException: Cannot init OpenTSDB connection!
	at com.alibaba.datax.plugin.reader.conn.OpenTSDBDump.getTSDB(OpenTSDBDump.java:41) ~[opentsdbreader-0.0.1-SNAPSHOT.jar:na]
	at com.alibaba.datax.plugin.reader.conn.OpenTSDBDump.dump(OpenTSDBDump.java:26) ~[opentsdbreader-0.0.1-SNAPSHOT.jar:na]
	at com.alibaba.datax.plugin.reader.conn.OpenTSDBConnection.sendDPs(OpenTSDBConnection.java:48) ~[opentsdbreader-0.0.1-SNAPSHOT.jar:na]
	at com.alibaba.datax.plugin.reader.opentsdbreader.OpenTSDBReader$Task.startRead(OpenTSDBReader.java:191) ~[opentsdbreader-0.0.1-SNAPSHOT.jar:na]
	... 2 common frames omitted
2020-04-08 10:44:23.120 [0-0-3009-reader] ERROR ReaderRunner - Reader runner Received Exceptions:
com.alibaba.datax.common.exception.DataXException: Code:[OpenTSDBReader-01], Description:[值非法].  - 获取或发送数据点的过程中出错! - java.lang.RuntimeException: Cannot init OpenTSDB connection!
	at com.alibaba.datax.plugin.reader.conn.OpenTSDBDump.getTSDB(OpenTSDBDump.java:41)
	at com.alibaba.datax.plugin.reader.conn.OpenTSDBDump.dump(OpenTSDBDump.java:26)
	at com.alibaba.datax.plugin.reader.conn.OpenTSDBConnection.sendDPs(OpenTSDBConnection.java:48)
	at com.alibaba.datax.plugin.reader.opentsdbreader.OpenTSDBReader$Task.startRead(OpenTSDBReader.java:191)
	at com.alibaba.datax.core.taskgroup.runner.ReaderRunner.run(ReaderRunner.java:57)
	at java.lang.Thread.run(Thread.java:748)

	at com.alibaba.datax.common.exception.DataXException.asDataXException(DataXException.java:33) ~[datax-common-0.0.1-SNAPSHOT.jar:na]
	at com.alibaba.datax.plugin.reader.opentsdbreader.OpenTSDBReader$Task.startRead(OpenTSDBReader.java:194) ~[opentsdbreader-0.0.1-SNAPSHOT.jar:na]
	at com.alibaba.datax.core.taskgroup.runner.ReaderRunner.run(ReaderRunner.java:57) ~[datax-core-0.0.1-SNAPSHOT.jar:na]
	at java.lang.Thread.run(Thread.java:748) [na:1.8.0_212]
Caused by: java.lang.RuntimeException: Cannot init OpenTSDB connection!
	at com.alibaba.datax.plugin.reader.conn.OpenTSDBDump.getTSDB(OpenTSDBDump.java:41) ~[opentsdbreader-0.0.1-SNAPSHOT.jar:na]
	at com.alibaba.datax.plugin.reader.conn.OpenTSDBDump.dump(OpenTSDBDump.java:26) ~[opentsdbreader-0.0.1-SNAPSHOT.jar:na]
	at com.alibaba.datax.plugin.reader.conn.OpenTSDBConnection.sendDPs(OpenTSDBConnection.java:48) ~[opentsdbreader-0.0.1-SNAPSHOT.jar:na]
	at com.alibaba.datax.plugin.reader.opentsdbreader.OpenTSDBReader$Task.startRead(OpenTSDBReader.java:191) ~[opentsdbreader-0.0.1-SNAPSHOT.jar:na]
	... 2 common fram
  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论

“相关推荐”对你有帮助么?

  • 非常没帮助
  • 没帮助
  • 一般
  • 有帮助
  • 非常有帮助
提交
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值