Intro
I have a Python project on a git repository. Everything works ok for most of the team members, we can sync the code and edit it without any pr
I faced similar issue with pyspark ( spark 2.1 ) & luigi.
Failed tries:
Unresolved reference pyspark could be fixed by adding the spark's python directory as Content Root in the project, but running the project as luigi task gave the same error.
Successfull steps
__init__.py file in the projectInclude following lines of code at the top of script
...
import sys
sys.path.append("/path/to/spark/python")
sys.path.append("/path/to/spark/python/lib")
...
// import pyspark files after above mentioned lines
and the Unresolved reference error issue was fixed both in pycharm & luigi task.