Spark Parallelize? (Could not find creator property with name 'id')

Question

What causes this Serialization error in Apache Spark 1.4.0 when calling:

sc.parallelize(strList, 4)

This exception is thrown:

Answer 1

@Interfector is correct. I ran into this issue also, here's a snippet from my sbt file and the 'dependencyOverrides' section which fixed it.

libraryDependencies ++= Seq(
  "com.amazonaws" % "amazon-kinesis-client" % "1.4.0",
  "org.apache.spark" %% "spark-core" % "1.4.0",
  "org.apache.spark" %% "spark-streaming" % "1.4.0",
  "org.apache.spark" %% "spark-streaming-kinesis-asl" % "1.4.0",
  "com.amazonaws" % "aws-java-sdk" % "1.10.2"
)

dependencyOverrides ++= Set(
  "com.fasterxml.jackson.core" % "jackson-databind" % "2.4.4"
)