spark 2.1.0 session config settings (pyspark)

Question

I am trying to overwrite the spark session/spark context default configs, but it is picking entire node/cluster resource.

 spark  = SparkSession.builder
            


        

Answer 1

Setting 'spark.driver.host' to 'localhost' in the config works for me

spark = SparkSession \
    .builder \
    .appName("MyApp") \
    .config("spark.driver.host", "localhost") \
    .getOrCreate()