Proper way to make a Spark Fat Jar using SBT

Question

I need a Fat Jar with Spark because I\'m creating a custom node for Knime. Basically it\'s a self-contained jar executed inside Knime and I assume a Fat Jar is the only way

Answer 1

I think the issue is with how you've setup assemblyMergeStrategy. Try this:

assemblyMergeStrategy in assembly := {
  case PathList("META-INF", xs @ _*) => MergeStrategy.discard
  case "application.conf"            => MergeStrategy.concat
  case "reference.conf"              => MergeStrategy.concat
  case x =>
    val baseStrategy = (assemblyMergeStrategy in assembly).value
    baseStrategy(x)
}