Running custom Java class in PySpark

Question

I\'m trying to run a custom HDFS reader class in PySpark. This class is written in Java and I need to access it from PySpark, either from the shell or with spark-submit.

Answer 1

Rather than --jars you should use --packages to import packages into your spark-submit action.