joblib.Parallel running through spyder hanging on Windows

问题

I'm running Python 3.5.1 on Windows Server 2013 at work. I have some embarrassingly parallel tasks that seem to work on Python 2.7 with basically the same code, but I am unable to figure out how to get it to run on Python 3.5.1.

I'm using Anaconda 2.4.1

The code looks like this... I've stripped it down to basically the minimum.

\
->main.py
\apackage\
->__init__.py
->amodule.py

Code for main.py

from tpackage import AClass

def go():
    x = AClass().AFunction()
    return x
if __name__ == '__main__':
    x = go()
    print(x)

Code for __init__.py

from .amodule import AClass

__all__ = ['AClass']

Code for amodule.py

from joblib import Parallel, delayed

class AClass(object):
    def AFunction(self):
        x = Parallel(n_jobs=2,verbose=100)(
            delayed(add1)(i) for i in range(10)
            )
        return x

def add1(x):   
    return x + 1

Does this have anything to do with the need for a if __name__ == '__main__': statement? I didn't think I would need this because the Parallel is protected inside a def statement already and should only run when the __main__ module is called, which should only happen once.

I should add that if I change n_jobs=1 in amodule.py, everything works fine.

Update:

So after further review, it appears that this probably something to do with spyder. I'm using spyder 2.3.8. When I have spyder execute this is a dedicated window, it works. But when it runs in the interactive IPython console, it fails. I can also run the program directly from the command line without problems.

Update 2:

After further review, this really has to do with IPython being in a different working directory than the *.py file. Get those lined up and this works.

回答1:

(Spyder dev here) If this problem is caused by runfile setting the working directory, you can prevent that to happen by going to the menu entry

Run > Configuration per file

(or pressing Ctrl+F6) and selecting the option called The current working directory.

Notes:

The run configuration is saved for each file and Spyder remembers it after restarts.
The answer has been updated for Spyder 3.2 and above.

来源：https://stackoverflow.com/questions/34821877/joblib-parallel-running-through-spyder-hanging-on-windows

标签

python

python-3.x

spyder

joblib