线上一个ETL Job不能跑了,报异常,这里为了说明问题简化表结构:

hive>desc void_t;
OK
x                       int                     None              
z                       void                    None

select* from void_t

确实会抛出异常:

14/03/0201:28:58 ERROR CliDriver: Failed with exceptionjava.io.IOException:org.apache.hadoop.hive.ql.metadata.HiveException: Errorevaluating x

看到这个异常很疑惑,和x字段有什么关系呢,继续看详细日志:

java.io.IOException:org.apache.hadoop.hive.ql.metadata.HiveException: Error evaluating x
        atorg.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:150)
        atorg.apache.hadoop.hive.ql.Driver.getResults(Driver.java:1412)
        at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:271)
        atorg.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
        atorg.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:413)
        at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:756)
        atorg.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614)
        atsun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        atsun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
        atsun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
        atjava.lang.reflect.Method.invoke(Method.java:597)
        atorg.apache.hadoop.util.RunJar.main(RunJar.java:208)
Causedby: org.apache.hadoop.hive.ql.metadata.HiveException: Error evaluating x
        atorg.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:80)
        atorg.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:502)
        at org.apache.hadoop.hive.ql.exec.Operator.forward(Operator.java:832)
        atorg.apache.hadoop.hive.ql.exec.TableScanOperator.processOp(TableScanOperator.java:90)
        atorg.apache.hadoop.hive.ql.exec.Operator.process(Operator.java:502)
        atorg.apache.hadoop.hive.ql.exec.FetchOperator.pushRow(FetchOperator.java:490)
        atorg.apache.hadoop.hive.ql.exec.FetchTask.fetch(FetchTask.java:136)
        ... 11 more
Causedby: java.lang.RuntimeException: Internal error: no LazyObject for VOID
        atorg.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyPrimitiveClass(LazyFactory.java:119)
        atorg.apache.hadoop.hive.serde2.lazy.LazyFactory.createLazyObject(LazyFactory.java:155)
        at org.apache.hadoop.hive.serde2.lazy.LazyStruct.parse(LazyStruct.java:108)
        atorg.apache.hadoop.hive.serde2.lazy.LazyStruct.getField(LazyStruct.java:190)
        atorg.apache.hadoop.hive.serde2.lazy.objectinspector.LazySimpleStructObjectInspector.getStructFieldData(LazySimpleStructObjectInspector.java:188)
        atorg.apache.hadoop.hive.serde2.objectinspector.DelegatedStructObjectInspector.getStructFieldData(DelegatedStructObjectInspector.java:79)
        atorg.apache.hadoop.hive.ql.exec.ExprNodeColumnEvaluator.evaluate(ExprNodeColumnEvaluator.java:98)
        atorg.apache.hadoop.hive.ql.exec.SelectOperator.processOp(SelectOperator.java:76)

看到这个noLazyObject for VOID才知道原来问题出现在这里,也就是字段z上;查看ETL Job里的Query发现里面一个建表的语句用到了create table xxx as select null as z from xxx这样的方式,进而生成了一个VOID类型的字段,但是Hive本身却无法处理该字段,在jira里确实也有这么一个unresolvedBug:HIVE-2615

Workaround也比较简单:1.先建表再insert select 2.ctascast(null as <type>) z来强制指定类型.