What is the right way to edit spark-env.sh before running spark-shell?

后端 未结 2 788
爱一瞬间的悲伤
爱一瞬间的悲伤 2020-12-30 15:41

I am running spark on my local windows machine. I am able to start spark shell successfully.

I want to edit the spark-env.sh file residing in conf/ folder. What is t

2条回答
  •  长情又很酷
    2020-12-30 15:57

    You must have to use export to add any configuration in *.sh file. So in spark-env.sh file use following example,

    export SPARK_MASTER_IP=192.165.5.1
    export SPARK_EXECUTOR_MEMORY=2g
    #OR export SPARK_EXECUTOR_MEMORY=2G
    

    No need to use double quotes for values.

提交回复
热议问题