Hadoop: java.lang.IncompatibleClassChangeError: Found interface org.apache.hadoop.mapreduce.JobContext, but class was expected

Question

My MapReduce jobs runs ok when assembled in Eclipse with all possible Hadoop and Hive jars included in Eclipse project as dependencies. (These are the jars that come with si

Answer 1

Hadoop has gone through a huge code refactoring from Hadoop 1.0 to Hadoop 2.0. One side effect is that code compiled against Hadoop 1.0 is not compatible with Hadoop 2.0 and vice-versa. However source code is mostly compatible and thus one just need to recompile code with target Hadoop distribution.

The exception "Found interface X, but class was expected" is very common when you're running code that is compiled for Hadoop 1.0 on Hadoop 2.0 or vice-versa.

You can find the correct hadoop version used in the cluster, then specify that hadoop version in the pom.xml file Build your project with the same version of hadoop used in the cluster and deploy it.