Hadoop “Failed to set setXIncludeAware(true) for parser” error and how to resolve it

转载 2013年12月03日 11:01:33

原文地址: http://caffeinbean.wordpress.com/2011/03/01/hadoop-failed-to-set-setxincludeawaretrue-for-parser-error-and-how-to-resolve-it/


Hadoop is a great piece of technology. But it’s not the technology that helps you solve the great problems. It’s the attitude you gain after absorbing the knowledge, and the courage to attack the problems.

For Hadoop, the “hello world” application is WordCount. Basically you feed a document with the assumption that it can be huge, the map reduce program outputs unique words and their counts. In real life however, the challenges you face is not as trivial. Some are not yet answered and subject to active exploration and development. Dependency injection is a hot topic for instance. But for this post I’ll focus on a specific problem and present you the solution.

If you ever have to deal with XML in map reduce environment, it’s possible that you get a stacktrace dump similar below.

1
ERROR conf.Configuration: Failed to set setXIncludeAware(true) for parser org.apache.xerces.jaxp.DocumentBuilderFactoryImpl@47315d34:java.lang.UnsupportedOperationException: This parser does not support specification "null" version "null"java.lang.UnsupportedOperationException: This parser does not support specification "null" version "null" at javax.xml.parsers.DocumentBuilderFactory.setXIncludeAware(DocumentBuilderFactory.java:590)   at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1054)   at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1030)  at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:980)    at org.apache.hadoop.conf.Configuration.set(Configuration.java:405) at org.apache.hadoop.conf.Configuration.setBoolean(Configuration.java:585)  at org.apache.hadoop.util.GenericOptionsParser.processGeneralOptions(GenericOptionsParser.java:290) at org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:375)   at org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:153)  at org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:138)  at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:59)    at my.job.MapReduce.main(MyJob.java:103)    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)    at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:156)

The reason is that the JDK supplied XML libraries are a bit out of date. In order to get rid of this error, you’ll need to both provide recent versions of Xalan and Xerces with you job configuration, which means you’ll need to make them available in your classpath.

If you’re using maven, (you are using maven for map reduce jobs right?) it’s just a couple of lines to include in the pom file.

1
2
3
4
5
6
7
8
9
10
<dependency>
    <groupId>xerces</groupId>
    <artifactId>xercesImpl</artifactId>
    <version>2.9.1</version>
</dependency>
<dependency>
    <groupId>xalan</groupId>
    <artifactId>xalan</artifactId>
    <version>2.7.1</version>
</dependency>

The versions for xalan are xerces are specific. You need to supply the versions listed or above.


相关文章推荐

Hadoop的Configuration类冲突问题

ERROR [org.apache.hadoop.conf.Configuration] - Failed to set setXIncludeAware(true) for parser org.a...

ERROR org.apache.hadoop.conf.Configuration: Failed to set setXIncludeAware(true) for parser

ERROR org.apache.hadoop.conf.Configuration: Failed to set setXIncludeAware(true) for parser org.apac...

Failed to set setXIncludeAware(true) for parser

解决方法:在JVM启劝参数中增加: -Djavax.xml.parsers.DocumentBuilderFactory=com.sun.org.apache.xerces.internal.jax...

Error in sitecustomize; set PYTHONVERBOSE for traceback: KeyError: 'PYTHONPATH'(python错误)

Error in sitecustomize; set PYTHONVERBOSE for traceback: KeyError: ‘PYTHONPATH’1.问题背景Mac下自带python2.7...

Android Studio Error:Failed to resolve: com.android.support:appcompat-v7:25.1.0

今天不知道为什么导入eclipse项目后就出现了错误,没导入之前是正常使用AS的 Error:(26, 13) Failed to resolve: com.android.support:appc...

Android Studio错误Error:(23, 17) Failed to resolve: junit:junit:4.12解决方案

问题描述: 在Android Studio创建项目之后,提示一个junit错误,百度了一下发现在Stackoverflow上有解答,但是都是英文的,我在这里自己发一个,希望大家支持。 解决方案: ...

Error:(24, 13) Failed to resolve: com.android.support:recyclerview-v7:20.+ <a href="install.m2.repo"

Android studio   报错    Error:(24, 13) Failed to resolve: com.android.support:recyclerview-v7:20.+ I...

Error:Failed to resolve: com.android.support:appcompat-v7:25.+d

当我新创建一个Android 项目时,提示一个错误如下: 错误提示说没有安装 Repository,然后我通过SDK Manager 下载 Android Support Repository ...

How to resolve ORA-19706 error when select from dblink

由于通过database link进行远程数据库的数据读取时,会自动同步当前库和远程库的SCN,老杨在文章中做过描述,并且提到: 而对于实际环境中,SCN的增加可能是几倍、几十倍甚至是上百倍,从而引...
  • halou90
  • halou90
  • 2015年12月04日 13:37
  • 816
内容举报
返回顶部
收藏助手
不良信息举报
您举报文章:Hadoop “Failed to set setXIncludeAware(true) for parser” error and how to resolve it
举报原因:
原因补充:

(最多只允许输入30个字)