Python multiprocessing throws error with argparse and pyinstaller

Question

In my project, I'm using argprse to pass arguments and somewhere in script I'm using multiprocessing to do rest of the calculations. Script is working fine if I call it from command prompt for ex.

"python complete_script.py --arg1=xy --arg2=yz" .

But after converting it to exe using Pyinstaller using command "pyinstaller --onefile complete_script.py" it throws

error

" error: unrecognized arguments: --multiprocessing-fork 1448"

Any suggestions how could I make this work. Or any other alternative. My goal is to create an exe application which I can call in other system where Python is not installed.

Here are the details of my workstation:

Platform: Windows 10
Python : 2.7.13 <installed using Anaconda>
multiprocessing : 0.70a1
argparse:   1.1

---

Copied from comment:

def main():
     main_parser = argparse.ArgumentParser()
     < added up arguments here>
    all_inputs = main_parser.parse_args()
    wrap_function(all_inputs)


def wrap_function(all_inputs):
    <Some calculation here >
   distribute_function(<input array for multiprocessing>)

def distribute_function(<input array>):
    pool = Pool(process = cpu_count)
    jobs = [pool.apply_async(target_functions, args = (i,) for i in input_array)]
    pool.close()

Answer 1

(A bit late but it can be useful for someone else in the future...)

I had the same problem, after some research I found this multiprocessing pyInstaller recipe that states:

When using the multiprocessing module, you must call

multiprocessing.freeze_support()

straight after the if __name__ == '__main__': line of the main module.

Please read the Python library manual about multiprocessing.freeze_support for more information.

Adding that line of code solved the problem for me.

Answer 2

I may be explaining the obvious, but you don't give us much information to work with.

python complete_script.py --arg1=xy --arg2=yz

This sort of call tells me that your parser is setup to accept at least these 2 arguments, ones flagged with '--arg1' and '--arg2'.

The error tells me that this parser (or maybe some other) is also seeing this string:

--multiprocessing-fork 1448

Possibly generated by the multiprocessing code. It would be good to see the usage part of the error, just to confirm which parser is complaining.

One of my first open source contributions to Python was to enhance the warnings about multiprocessing on Windows.

https://docs.python.org/2/library/multiprocessing.html#windows

Is your parser protected by a if __name__ block? Should this particular parser be called when run in a fork? You probably designed the parser to work when the program is called as a standalone script. But when happens when it is imported?