How to write JDBC Sink for Spark Structured Streaming [SparkException: Task not serializable]?

Question

I need a JDBC sink for my spark structured streaming data frame. At the moment, as far as I know DataFrame’s API lacks writeStream to JDBC implementation (neith

Answer 1

Just define JDBCSink in a separated file rather than defining it as an inner class which may capture the outer reference.