Why does format(“kafka”) fail with “Failed to find data source: kafka.” (even with uber-jar)?

Question

I use HDP-2.6.3.0 with Spark2 package 2.2.0.

I\'m trying to write a Kafka consumer, using the Structured Streaming API, but I\'m getting the following error after su

Answer 1

For uber-jar, adding ServicesResourceTransformer to shade-plugin works for me.