# Hadoop “Failed to set setXIncludeAware(true) for parser” error and how to resolve it

Hadoop is a great piece of technology. But it’s not the technology that helps you solve the great problems. It’s the attitude you gain after absorbing the knowledge, and the courage to attack the problems.

For Hadoop, the “hello world” application is WordCount. Basically you feed a document with the assumption that it can be huge, the map reduce program outputs unique words and their counts. In real life however, the challenges you face is not as trivial. Some are not yet answered and subject to active exploration and development. Dependency injection is a hot topic for instance. But for this post I’ll focus on a specific problem and present you the solution.

If you ever have to deal with XML in map reduce environment, it’s possible that you get a stacktrace dump similar below.

 1 ERROR conf.Configuration: Failed to set setXIncludeAware(true) for parser org.apache.xerces.jaxp.DocumentBuilderFactoryImpl@47315d34:java.lang.UnsupportedOperationException: This parser does not support specification "null" version "null"java.lang.UnsupportedOperationException: This parser does not support specification "null" version "null" at javax.xml.parsers.DocumentBuilderFactory.setXIncludeAware(DocumentBuilderFactory.java:590)   at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1054)   at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1030)  at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:980)    at org.apache.hadoop.conf.Configuration.set(Configuration.java:405) at org.apache.hadoop.conf.Configuration.setBoolean(Configuration.java:585)  at org.apache.hadoop.util.GenericOptionsParser.processGeneralOptions(GenericOptionsParser.java:290) at org.apache.hadoop.util.GenericOptionsParser.parseGeneralOptions(GenericOptionsParser.java:375)   at org.apache.hadoop.util.GenericOptionsParser.(GenericOptionsParser.java:153)  at org.apache.hadoop.util.GenericOptionsParser.(GenericOptionsParser.java:138)  at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:59)    at my.job.MapReduce.main(MyJob.java:103)    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)  at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)    at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.util.RunJar.main(RunJar.java:156)

The reason is that the JDK supplied XML libraries are a bit out of date. In order to get rid of this error, you’ll need to both provide recent versions of Xalan and Xerces with you job configuration, which means you’ll need to make them available in your classpath.

If you’re using maven, (you are using maven for map reduce jobs right?) it’s just a couple of lines to include in the pom file.

 1 2 3 4 5 6 7 8 9 10      xerces     xercesImpl     2.9.1       xalan     xalan     2.7.1 

The versions for xalan are xerces are specific. You need to supply the versions listed or above.

• 本文已收录于以下专栏：

举报原因： 您举报文章：Hadoop “Failed to set setXIncludeAware(true) for parser” error and how to resolve it 色情 政治 抄袭 广告 招聘 骂人 其他 (最多只允许输入30个字)