Apache Hadoop setXIncludeAware UnsupportedOperationException

Question

I'm trying to get Apache Hadoop 1.21 running, but I'm getting this exception:

Failed to set setXIncludeAware(true) for parser org.apache.xerces.jaxp.DocumentBuilderFactoryImpl
org.apache.xerces.jaxp.DocumentBuilderFactoryImpl@2662e5cf:java.lang.UnsupportedOperationException

full stack trace:

13/10/17 17:22:52 ERROR conf.Configuration: Failed to set setXIncludeAware(true) for parser org.apache.xerces.jaxp.DocumentBuilderFactoryImpl@2662e5cf:java.lang.UnsupportedOperationException:  setXIncludeAware is not supported on this JAXP implementation or earlier: class org.apache.xerces.jaxp.DocumentBuilderFactoryImpl
java.lang.UnsupportedOperationException:  setXIncludeAware is not supported on this JAXP implementation or earlier: class org.apache.xerces.jaxp.DocumentBuilderFactoryImpl
    at javax.xml.parsers.DocumentBuilderFactory.setXIncludeAware(DocumentBuilderFactory.java:589)
    at org.apache.hadoop.conf.Configuration.loadResource(Configuration.java:1131)
    at org.apache.hadoop.conf.Configuration.loadResources(Configuration.java:1107)
    at org.apache.hadoop.conf.Configuration.getProps(Configuration.java:1053)
    at org.apache.hadoop.conf.Configuration.get(Configuration.java:460)
    at org.apache.hadoop.fs.FileSystem.getDefaultUri(FileSystem.java:132)
    at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:124)
    at main.JobExecutor.executeModelCreation(JobExecutor.java:223)
    at main.JobExecutor.main(JobExecutor.java:256)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:601)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:160)

I have been looking for the solution for almost 3 days, I have found several websites (like this one: Hadoop “Failed to set setXIncludeAware(true) for parser” error and how to resolve it), which suggested add xerces and xalan to maven dependencies. There were also other websites, which suggested pretty much opposite - remove all xerces references from classpath. But non of the suggested solutions is working :(

I've also tried to put:

System.setProperty("javax.xml.parsers.DocumentBuilderFactory", "com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderFactoryImpl"); 

to my code. But it didn't help either :(

I'm using Apache Hadoop 1.21 as I said and JDK 1.7.0-17.

Answer 1

In my case it was outdated Maven component dependency on xerces-impl 2.4.0 (due to mockrunner 1.0.3 used on test phase). Following managed dependency added to <dependencyManagemen> section helped.

<dependencyManagement>
    <dependencies>
        <dependency>
            <groupId>xerces</groupId>
            <artifactId>xercesImpl</artifactId>
            <version>2.11.0</version>
        </dependency>
    </dependencies>
</dependencyManagement>

Hadoop version was 2.3.0. Here is another article which describes similar situation.

Answer 2

Try passing -Djavax.xml.parsers.DocumentBuilderFactory=com.sun.org.apache.xerces.internal.jaxp.DocumentBuilderFactoryImpl

in as a command ling arg. Maybe you're not setting the property early enough?