org.apache.hive.com.esotericsoftware.kryo.kryoexception: encountered unregistered class id 错误解决办法

执行hive 任务的时候 有些任务会报下列错误 hive 0.14 + 版本才会有这个问题,

任务重做之后可能又会成功。

1.错误信息:

hdfs://nameservice1/tmp/hive/dbs/9c29873a-664f-45a4-87f5-aab6601b9661/hive_2015-09-22_14-51-49_927_648109380558598376-11/-mr-10066/129d4884-d21a-44f2-970c-454148932292/map.xml: 
org.apache.hive.com.esotericsoftware.kryo.kryoexception: encountered unregistered class id: 109
serialization trace:
parentoperators (org.apache.hadoop.hive.ql.exec.tablescanoperator)
aliastowork (org.apache.hadoop.hive.ql.plan.mapwork)
at org.apache.hadoop.hive.ql.exec.utilities.getbasework(utilities.java:450)
at org.apache.hadoop.hive.ql.exec.utilities.getmapwork(utilities.java:296)
at org.apache.hadoop.hive.ql.io.hiveinputformat.init(hiveinputformat.java:268)
at org.apache.hadoop.hive.ql.io.hiveinputformat.pushprojectionsandfilters(hiveinputformat.java:484)
at org.apache.hadoop.hive.ql.io.hiveinputformat.pushprojectionsandfilters(hiveinputformat.java:477)
at org.apache.hadoop.hive.ql.io.combinehiveinputformat.getrecordreader(combinehiveinputformat.java:715)
at org.apache.hadoop.mapred.maptask$trackedrecordreader.<init>(maptask.java:169)
at org.apache.hadoop.mapred.maptask.runoldmapper(maptask.java:432)
at org.apache.hadoop.mapred.maptask.run(maptask.java:343)
at org.apache.hadoop.mapred.yarnchild$2.run(yarnchild.java:163)
at java.security.accesscontroller.doprivileged(native method)
at javax.security.auth.subject.doas(subject.java:415)
at org.apache.hadoop.security.usergroupinformation.doas(usergroupinformation.java:1671)
at org.apache.hadoop.mapred.yarnchild.main(yarnchild.java:158)
caused by: org.apache.hive.com.esotericsoftware.kryo.kryoexception: encountered unregistered class id: 109
serialization trace:
parentoperators (org.apache.hadoop.hive.ql.exec.tablescanoperator)
aliastowork (org.apache.hadoop.hive.ql.plan.mapwork)
at org.apache.hive.com.esotericsoftware.kryo.util.defaultclassresolver.readclass(defaultclassresolver.java:119)
at org.apache.hive.com.esotericsoftware.kryo.kryo.readclass(kryo.java:656)
at org.apache.hive.com.esotericsoftware.kryo.serializers.objectfield.read(objectfield.java:99)
at org.apache.hive.com.esotericsoftware.kryo.serializers.fieldserializer.read(fieldserializer.java:507)
at org.apache.hive.com.esotericsoftware.kryo.kryo.readclassandobject(kryo.java:776)
at org.apache.hive.com.esotericsoftware.kryo.serializers.mapserializer.read(mapserializer.java:139)
at org.apache.hive.com.esotericsoftware.kryo.serializers.mapserializer.read(mapserializer.java:17)
at org.apache.hive.com.esotericsoftware.kryo.kryo.readobject(kryo.java:694)
at org.apache.hive.com.esotericsoftware.kryo.serializers.objectfield.read(objectfield.java:106)
at org.apache.hive.com.esotericsoftware.kryo.serializers.fieldserializer.read(fieldserializer.java:507)
at org.apache.hive.com.esotericsoftware.kryo.kryo.readobject(kryo.java:672)
at org.apache.hadoop.hive.ql.exec.utilities.deserializeobjectbykryo(utilities.java:1069)
at org.apache.hadoop.hive.ql.exec.utilities.deserializeplan(utilities.java:960)
at org.apache.hadoop.hive.ql.exec.utilities.deserializeplan(utilities.java:974)
at org.apache.hadoop.hive.ql.exec.utilities.getbasework(utilities.java:416)
... 13 more

ended job = job_1442370393660_27415 with errors
error during job, obtaining debugging information...
examining task id: task_1442370393660_27415_m_000004 (and more) from job job_1442370393660_27415
examining task id: task_1442370393660_27415_m_000002 (and more) from job job_1442370393660_2741
examining task id: task_1442370393660_27415_m_000001 (and more) from job job_1442370393660_27415

2.参数分析:

<property>
    <name>hive.plan.serialization.format</name>
    <value>kryo</value>
    <description>
      Query plan format serialization between client and task nodes. 
      Two supported values are : kryo and javaXML. Kryo is default.
    </description>
  </property>

执行计划相关的参数 默认kryo
在客户端设置为 javaXML,如果是用CM安装的则在客户端高级选项里面添加为 javaXML 则可。

3.临时解决办法:

在经常报错的hive 前面把参数设置为javaXML 要是失败的任务比较少 则再相应的任务里面用session级别设置下就可以了。

set  hive.plan.serialization.format=javaXML;
  • 0
    点赞
  • 0
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值