Difference between SparkContext, JavaSparkContext, SQLContext, and SparkSession?

Question

1. What is the difference between SparkContext, JavaSparkContext, SQLContext and SparkSession?
2. Is there any m

Answer 1

Spark Context is Class in Spark API which is the first stage to build the spark application. Functionality of the spark context is to create memory in RAM we call this as driver memory, allocation of number of executers and cores in short its all about the cluster management. Spark Context can be used to create RDD and shared variables. To access this we need to create object of it.

This way we can create Spark Context :: var sc=new SparkContext()

Spark Session this is new Object added since spark 2.x which is replacement of Sql Context and Hive Context. Earlier we had two options like one is Sql Context which is way to do sql operation on Dataframe and second is Hive Context which manage the Hive connectivity related stuff and fetch/insert the data from/to the hive tables.

Since 2.x came We can create SparkSession for the SQL operation on Dataframe and if you have any Hive related work just call the Method enablehivesupport() then you can use the SparkSession for the both Dataframe and hive related SQL operations.

This way we can create SparkSession for Sql operation on Dataframe

val sparksession=SparkSession.builder().getOrCreate();

Second way is to create SparkSession for Sql operation on Dataframe as well as Hive Operation.

val sparkSession=SparkSession.builder().enableHiveSupport().getOrCreate()