How do I install pyspark for use in standalone scripts?

Question

I\'m am trying to use Spark with Python. I installed the Spark 1.0.2 for Hadoop 2 binary distribution from the downloads page. I can run through the quickstart examples in P

Answer 1

Spark-2.2.0 onwards use pip install pyspark to install pyspark in your machine.

For older versions refer following steps. Add Pyspark lib in Python path in the bashrc

export PYTHONPATH=$SPARK_HOME/python/:$PYTHONPATH

also don't forget to set up the SPARK_HOME. PySpark depends the py4j Python package. So install that as follows

pip install py4j

For more details about stand alone PySpark application refer this post