Spark 2.0.0 twitter streaming driver is no longer available

Question

During migration from spark 1.6.2 to spark 2.0.0 appeared that package org.apache.spark.streaming.twitter has been removed and twitter streaming is no longer available as we

Answer 1

Twitter (and some other) driver support has been removed in Spark 2.0.

You can see it in the removal section of the Release Notes:

Removals

The following features have been removed in Spark 2.0:

• Less frequently used streaming connectors, including Twitter, Akka, MQTT, ZeroMQ

They have been extracted as a separate package under the Bahir Project. The twitter extension, streaming-twitter, can be found via:

sbt:

libraryDependencies += "org.apache.bahir" %% "spark-streaming-twitter" % "2.0.0"

Maven:

  org.apache.bahir
  spark-streaming-twitter_2.11
  2.0.0-preview

More on that (thanks to @IvanShulak) in the Mailing List

Edit:

For Spark 2.0.1, use:

libraryDependencies += "org.apache.bahir" %% "spark-streaming-twitter" % "2.0.1"