Spark Unable to find JDBC Driver

Question

So I\'ve been using sbt with assembly to package all my dependencies into a single jar for my spark jobs. I\'ve got several jobs where I was using c3p0 to setu

Answer 1

I had the same problem running jobs over a Mesos cluster in cluster mode.

To use a JDBC driver is necessary to add the dependency to the system classpath not to the framework classpath. I only found the way of doing it by adding the dependency in the file spark-defaults.conf in every instance of the cluster.

The properties to add are spark.driver.extraClassPath and spark.executor.extraClassPath and the path must be in the local file system.