Hadoop: java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.JobContext, but class was expected

Question

My MapReduce jobs runs ok when assembled in Eclipse with all possible Hadoop and Hive jars included in Eclipse project as dependencies. (These are the jars that come with si

Answer 1

Obviously, you have versions incompatibility between you Hadoop and Hive versions. You need to upgrade (or downgrade) your Hadoop version or Hive version.

This is due the incompatibility between Hadoop 1 and Hadoop 2.

Answer 2

Hadoop has gone through a huge code refactoring from Hadoop 1.0 to Hadoop 2.0. One side effect is that code compiled against Hadoop 1.0 is not compatible with Hadoop 2.0 and vice-versa. However source code is mostly compatible and thus one just need to recompile code with target Hadoop distribution.

The exception "Found interface X, but class was expected" is very common when you're running code that is compiled for Hadoop 1.0 on Hadoop 2.0 or vice-versa.

You can find the correct hadoop version used in the cluster, then specify that hadoop version in the pom.xml file Build your project with the same version of hadoop used in the cluster and deploy it.

Answer 3

You need to recompile "hcatalog-core" to support Hadoop 2.0.0. Currently "hcatalog-core" only supports Hadoop 1.0

Answer 4

Even I ran through this problem. Was trying use HCatMultipleInputs with hive-hcatalog-core-0.13.0.jar. We are using hadoop 2.5.1.

The following code change helped me fix the issue:

//JobContext ctx = new JobContext(conf,jobContext.getJobID());
JobContext ctx = new Job(conf);

Answer 5

Look for entries like this

<dependency>
            <groupId>org.apache.hadoop</groupId>
            <artifactId>hadoop-core</artifactId>
            <version>1.2.1</version>
</dependency>

in your pom.xml. These define the hadoop version to use. Change them or remove them as per your requirements.