Spark 2.3 java.lang.NoSuchMethodError: io.netty.buffer.PooledByteBufAllocator.metric

Question

SPARK 2.3 is throwing following exception. Can anyone please help!! I tried adding the JARs

308 [Driver] ERROR org.apache.spark.deploy.yarn.ApplicationMaster -

Answer 1

This issue plagues due to mismatch of the version that Hadoop and Spark are compiled on for Netty. So you can follow this.

Similar Issue , solved by manually compiling the Spark by using specific version of Netty

And the other one as recommended by Suhas , by copying the content of SPARK_HOME/jars folder to the various lib folder or only the one in yarn inside HADOOP_HOME/share/hadoop solves the problem also. But it's a dirty fix. So maybe use latest version of both or manually compile them.