Geting error NoClassDefFoundError: org.apache.spark.internal.Logging on Kafka Spark Stream

Question

I am getting an error on run time by running below java code,

Is there any dependencies need to include for logging like log4js or anything?

Why this error

Answer 1

import org.apache.spark.Logging, it is available in Spark version 1.5.2 or later, so I would recommend you to use 1.5.2 or a later version of spark.

Which spark version you are using?

There is another dependency that solves this problem, and is compatible with Spark 2.x.

For SBT, use this dependency:

"org.apache.bahir" %% "spark-streaming-twitter" % "2.0.0"

Answer 2

Problem got resolved by setting up mentioned above dependencies into below order,

<dependency>
            <groupId>log4j</groupId>
            <artifactId>log4j</artifactId>
            <version>1.2.17</version>
        </dependency>
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-core_2.10</artifactId>
            <version>2.0.0</version>
        </dependency>
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-streaming-kafka-0-10_2.10</artifactId>
            <version>2.0.0</version>
        </dependency>
        <dependency>
            <groupId>org.apache.spark</groupId>
            <artifactId>spark-mllib_2.10</artifactId>
            <version>2.0.0</version>
        </dependency>