How to run a script in PySpark

Question

I\'m trying to run a script in the pyspark environment but so far I haven\'t been able to. How can I run a script like python script.py but in pyspark? Thanks

Answer 1

pyspark 2.0 and later execute script file in environment variable PYTHONSTARTUP, so you can run:

PYTHONSTARTUP=code.py pyspark

Compared to spark-submit answer this is useful for running initialization code before using the interactive pyspark shell.