DataX实现从hive到Oracle抽取数据报错:
2020-07-08 16:43:48.011 [0-0-0-reader] ERROR ReaderRunner - Reader runner Received Exceptions:
com.alibaba.datax.common.exception.DataXException: Code:[HdfsReader-00], Description:[您配置的值不合法.]. - 请确认您所读取的列配置正确!columnIndexMax 小于0,column:[]
at com.alibaba.datax.common.exception.DataXException.asDataXException(DataXException.java:26) ~[datax-common-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.reader.hdfsreader.DFSUtil.orcFileStartRead(DFSUtil.java:362) ~[hdfsreader-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.plugin.reader.hdfsreader.HdfsReader$Task.startRead(HdfsReader.java:269) ~[hdfsreader-0.0.1-SNAPSHOT.jar:na]
at com.alibaba.datax.core.taskgroup.runner.ReaderRunner.run(ReaderRunner.java:57) ~[datax-core-0.0.1-SNAPSHOT.jar:na]
at java.lang.Thread.run(Thread.java:748) [na:1.8.0_162]
Exception in thread "taskGroup-0" com.alibaba.datax.common.exception.DataXException: Code:[HdfsReader-00], Description:[您配置的值不合法.]. - 请确认您所读取的列配置正确!columnIndexMax 小于0,column:[]
at com.alibaba.datax.common.exception.DataXException.asDataXException(DataXException.java:26)
at com.alibaba.datax.plugin.reader.hdfsreader.DFSUtil.orcFileStartRead(DFSUtil.java:362)
at com.alibaba.datax.plugin.reader.hdfsreader.HdfsReader$Task.startRead(HdfsReader.java:269)
at com.alibaba.datax.core.taskgroup.runner.ReaderRunner.run(ReaderRunner.java:57)
at java.lang.Thread.run(Thread.java:748)
原因是用到了 “column”: ["*"]
"reader": {
"name": "hdfsreader",
"parameter": {
"path": "/user/hive/warehouse/mds.db/d_meter_stats",
"defaultFS": "hdfs://master:8020",
"column": ["*"],
"fileType": "orc",
"encoding": "UTF-8",
"fieldDelimiter": ","
}
},
官网有说:
默认情况下,用户可以全部按照String类型读取数据,配置如下:
"column": ["*"]
也就是说 “column”: ["*"] 是所有字段全为string的时候才可以用。