Configuring Apache Spark Logging with Scala and logback

Question

I am very confused with setting up logging with Apache Spark. Apache spark used Log4j for logging and it generates huge amount of log data. Is there a way to setup log4j fo

Answer 1

I have used the following imports:

import org.slf4j.Logger
import org.slf4j.LoggerFactory

Sample code as shown below.

Object SparkCode {

val logger = LoggerFactory.getLogger(this.getClass.getName)

  def main(args: Array[String]) {

  logger.info("Connection Started . ")

  // Create Spark Context and go on..

  }

}

And you are sorted.