Is there a simple process-based parallel map for python?

Question

I\'m looking for a simple process-based parallel map for python, that is, a function

parmap(function,[data])

that would run function on eac

Answer 1

I know this is an old post, but just in case, I wrote a tool to make this super, super easy called parmapper (I actually call it parmap in my use but the name was taken).

It handles a lot of the setup and deconstruction of processes and adds tons of features. In rough order of importance

• Can take lambda and other unpickleable functions
• Can apply starmap and other similar call methods to make it very easy to directly use.
• Can split amongst both threads and/or processes
• Includes features such as progress bars

It does incur a small cost but for most uses, that is negligible.

I hope you find it useful.

(Note: It, like map in Python 3+, returns an iterable so if you expect all results to pass through it immediately, use list())