Is it possible to use json4s 3.2.11 with Spark 1.3.0?

Question

Spark has a dependency on json4s 3.2.10, but this version has several bugs and I need to use 3.2.11. I added json4s-native 3.2.11 dependency to build.sbt and everything comp

Answer 1

I asked the same question in the Spark User Mailing List, and got two answers how to make it work:

1. Use spark.driver.userClassPathFirst=true and spark.executor.userClassPathFirst=true options, but it works only in Spark 1.3 and probably will require some other modifications like excluding Scala classes from your build.

2. Rebuild Spark with json4s 3.2.11 version (you can change it in core/pom.xml)

Both work fine, I prefered the second one.