Unable to run pyspark

Question

I installed Spark on Windows, and I\'m unable to start pyspark. When I type in c:\\Spark\\bin\\pyspark, I get the following error:

Answer 1

I wanted to extend on Indrajeet's answer, since he mentioned line numbers instead of the exact location of the code. Please see this in addition to his answer for further clarification.

cls = _old_namedtuple(*args, **kwargs)
is the line that was changed being referred in his answer

def _hijack_namedtuple():
""" Hack namedtuple() to make it picklable """
# hijack only one time
if hasattr(collections.namedtuple, "__hijack"):
    return

global _old_namedtuple  # or it will put in closure

def _copy_func(f):
    return types.FunctionType(f.__code__, f.__globals__, f.__name__,
                              f.__defaults__, f.__closure__)

_old_namedtuple = _copy_func(collections.namedtuple)

def namedtuple(*args, **kwargs):
    # cls = _old_namedtuple(*args, **kwargs)
    cls = _old_namedtuple(*args, **kwargs, verbose=False, rename=False, module=None)
    return _hack_namedtuple(cls)

!!! EDIT 6th Mar 2017!! This did fix the original issue, but I don't think this will make spark 2.1 compatible with 3.6 yet, there were more collisions further down. As a result I used conda to create a python 35 virtual environment and it worked like a charm.

(Windows, assuming you have env variables in place)

>conda create -n py35 python=3.5
>activate py35 
>pyspark