How /where are the jvm options used by the pyspark script when launching the jvm it connects to specified?
I am specifically interested in specifying jvm debugging options e.g.
-agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=5005
Thanks.
pyspark uses the bin/spark-class script to start the client that you see running in your terminal / console. You can just append whatever options necessary to JAVA_OPTS:
JAVA_OPTS="$JAVA_OPTS -Xmx=2g -Xms=1g -agentlib:jdwp=transport=dt_socket,server=y..."
来源：https://stackoverflow.com/questions/29478112/specify-options-for-the-jvm-launched-by-pyspark