How to run a script in PySpark

Question

I\'m trying to run a script in the pyspark environment but so far I haven\'t been able to. How can I run a script like python script.py but in pyspark? Thanks

Answer 1

You can execute "script.py" as follows

pyspark < script.py

or

# if you want to run pyspark in yarn cluster
pyspark --master yarn < script.py