hive 中出现struct 结构化的问题

最新推荐文章于 2024-05-29 09:15:00 发布

bill_live

最新推荐文章于 2024-05-29 09:15:00 发布

阅读量353

点赞数

文章标签：大数据 java 数据结构与算法

原文链接：http://www.cnblogs.com/MengYan-LongYou/p/3210189.html

版权

如果你使用udf，udaf，udtf中的某一个并且查询日志中出现如下之类的struct错误

java.lang.RuntimeException: Error in configuring object
	at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:104)
	at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:74)
	at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
	at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:128)
	at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:370)
	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:324)
	at org.apache.hadoop.mapred.Child.main(Child.java:165)
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:99)
	... 6 more
Caused by: java.lang.RuntimeException: Error in configuring object
	at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:104)
	at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:74)
	at org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:64)
	at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:128)
	at org.apache.hadoop.mapred.MapRunner.configure(MapRunner.java:34)
	... 11 more
Caused by: java.lang.reflect.InvocationTargetException
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
	at java.lang.reflect.Method.invoke(Method.java:597)
	at org.apache.hadoop.util.ReflectionUtils.setJobConf(ReflectionUtils.java:99)
	... 15 more
Caused by: java.lang.RuntimeException: Map operator initialization failed
	at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:130)
	... 20 more
Caused by: java.lang.RuntimeException: java.lang.IllegalArgumentException: Error: name expected at the position 67 of 'struct,mapl:map>,int>>,mapr:map>,int>>,sum:int>' but '>' is found.
	at org.apache.hadoop.hive.ql.exec.ReduceSinkOperator.initializeOp(ReduceSinkOperator.java:129)
	at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:351)
	at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:427)
	at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:383)
	at org.apache.hadoop.hive.ql.exec.GroupByOperator.initializeOp(GroupByOperator.java:368)
	at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:351)
	at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:427)
	at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:383)
	at org.apache.hadoop.hive.ql.exec.SelectOperator.initializeOp(SelectOperator.java:62)
	at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:351)
	at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:427)
	at org.apache.hadoop.hive.ql.exec.Operator.initializeChildren(Operator.java:383)
	at org.apache.hadoop.hive.ql.exec.Operator.initializeOp(Operator.java:368)
	at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:351)
	at org.apache.hadoop.hive.ql.exec.MapOperator.initializeOp(MapOperator.java:358)
	at org.apache.hadoop.hive.ql.exec.Operator.initialize(Operator.java:351)
	at org.apache.hadoop.hive.ql.exec.ExecMapper.configure(ExecMapper.java:94)

那么请注意一下编写的函数中是否如果是复合型数据结构，那么一定不要出现Character

转载于:https://www.cnblogs.com/MengYan-LongYou/p/3210189.html