Hadoop “Failed to set setXIncludeAware(true) for parser” error and how to resolve it

转载 2013年12月03日 11:01:33

原文地址: http://caffeinbean.wordpress.com/2011/03/01/hadoop-failed-to-set-setxincludeawaretrue-for-parser-error-and-how-to-resolve-it/


Hadoop is a great piece of technology. But it’s not the technology that helps you solve the great problems. It’s the attitude you gain after absorbing the knowledge, and the courage to attack the problems.

For Hadoop, the “hello world” application is WordCount. Basically you feed a document with the assumption that it can be huge, the map reduce program outputs unique words and their counts. In real life however, the challenges you face is not as trivial. Some are not yet answered and subject to active exploration and development. Dependency injection is a hot topic for instance. But for this post I’ll focus on a specific problem and present you the solution.

If you ever have to deal with XML in map reduce environment, it’s possible that you get a stacktrace dump similar below.

1
ERROR conf.Configuration: Failed to set setXIncludeAware(true) for parser org.apache.xerces.jaxp.DocumentBuilderFactoryImpl@47315d34:java.lang.UnsupportedOperationException: This parser does not support specification "null" version "null"java.lang.UnsupportedOperationException: This parser does not support specification "null" version "null" at javax.xml.parsers.DocumentBuilderFactory.setXIncludeAware(DocumentBuilderFactory.java:590)   at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1054)   at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1030)  at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:980)    at org.apache.hadoop.conf.Configuration.set(Configuration.java:405) at org.apache.hadoop.conf.Configuration.setBoolean(Configuration.java:585)  at org.apache.hadoop.util.GenericOptionsParser.processGeneralOptions(GenericOptionsParser.java:290) at org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:375)   at org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:153)  at org.apache.hadoop.util.GenericOptionsParser.<init>(GenericOptionsParser.java:138)  at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:59)    at my.job.MapReduce.main(MyJob.java:103)    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)    at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:156)

The reason is that the JDK supplied XML libraries are a bit out of date. In order to get rid of this error, you’ll need to both provide recent versions of Xalan and Xerces with you job configuration, which means you’ll need to make them available in your classpath.

If you’re using maven, (you are using maven for map reduce jobs right?) it’s just a couple of lines to include in the pom file.

1
2
3
4
5
6
7
8
9
10
<dependency>
    <groupId>xerces</groupId>
    <artifactId>xercesImpl</artifactId>
    <version>2.9.1</version>
</dependency>
<dependency>
    <groupId>xalan</groupId>
    <artifactId>xalan</artifactId>
    <version>2.7.1</version>
</dependency>

The versions for xalan are xerces are specific. You need to supply the versions listed or above.


ERROR org.apache.hadoop.conf.Configuration: Failed to set setXIncludeAware(true) for parser

ERROR org.apache.hadoop.conf.Configuration: Failed to set setXIncludeAware(true) for parser org.apac...
  • shufanshijie
  • shufanshijie
  • 2013年04月20日 13:06
  • 1873

Failed to set setXIncludeAware(true) for parser

解决方法:在JVM启劝参数中增加: -Djavax.xml.parsers.DocumentBuilderFactory=com.sun.org.apache.xerces.internal.jax...
  • mooninday
  • mooninday
  • 2013年09月24日 11:15
  • 1072

hadoop环境配置过程中可能遇到问题的解决方案

Failed to set setXIncludeAware(true) for parser 遇到此问题一般是jar包冲突的问题。一种情况是我们向java的lib目录添加我们自己的jar包导致ha...
  • yutianzuijin
  • yutianzuijin
  • 2013年07月24日 20:22
  • 35215

hadoop一些常见报错的解决方案

hadoop环境配置过程中可能遇到问题的解决方案
  • xiaolewennofollow
  • xiaolewennofollow
  • 2015年07月15日 17:03
  • 2586

Failed to set setXIncludeAware(true) for parser com.caucho.xml.parsers.XmlDocumentBuilderFactory@d83

2015-11-24 13:44:27,027 [main] ERROR conf.Configuration (Configuration.java:2370) - Failed to se...
  • blackice1015
  • blackice1015
  • 2015年11月24日 13:45
  • 673

Android Studio - Error: Failed to resolve: com.android.support.constraint:constraint-layout:1.0.0-al

转载自:http://stackoverflow.com/questions/40160968/android-studio-error-failed-to-resolve-com-android-s...
  • u011404670
  • u011404670
  • 2017年02月04日 17:12
  • 6821

hadoop的安装

Hadoop的安装根据之前 的BLOG介绍,在CYGWIN下安装Hadoop时出现了几个问题。 1)安装完格式化时提示找不到winutils文件。这个还好解决,找到一个这个文件,或者从Hadoop的...
  • wangato
  • wangato
  • 2017年03月20日 18:15
  • 153

Hadoop “Failed to set setXIncludeAware(true) for parser” error and how to resolve it

原文地址: http://caffeinbean.wordpress.com/2011/03/01/hadoop-failed-to-set-setxincludeawaretrue-for-pars...
  • harjolgoo
  • harjolgoo
  • 2013年12月03日 11:01
  • 1056

Error:Failed to resolve: com.android.databinding:adapters:1.2.1

使用android studio编写dataBinding数据绑定的时候,出现错误Error:Failed to resolve: com.android.databinding:library:1....
  • u011028777
  • u011028777
  • 2017年03月06日 12:28
  • 538

Android Studio Error:Failed to resolve: com.android.support:appcompat-v7:25.1.0

今天不知道为什么导入eclipse项目后就出现了错误,没导入之前是正常使用AS的 Error:(26, 13) Failed to resolve: com.android.support:appc...
  • qq_33098039
  • qq_33098039
  • 2017年07月04日 23:35
  • 5261
内容举报
返回顶部
收藏助手
不良信息举报
您举报文章:Hadoop “Failed to set setXIncludeAware(true) for parser” error and how to resolve it
举报原因:
原因补充:

(最多只允许输入30个字)