Multiprocessing with multiple arguments to function in Python 2.7

Question

I\'m trying to implement multiprocessing to speed up a replication loop, but cannot get it to work in Python27. This is a very simplified version of my program, based on the

Answer 1

There's a fork of multiprocessing called pathos (note: use the version on github) that doesn't need starmap or helpers or all of that other stuff -- the map functions mirror the API for python's map, thus map can take multiple arguments. With pathos, you can also generally do multiprocessing in the interpreter, instead of being stuck in the __main__ block. pathos is due for a release, after some mild updating -- mostly conversion to python 3.x.

  Python 2.7.5 (default, Sep 30 2013, 20:15:49) 
  [GCC 4.2.1 (Apple Inc. build 5566)] on darwin
  Type "help", "copyright", "credits" or "license" for more information.
  >>> from pathos.multiprocessing import ProcessingPool    
  >>> pool = ProcessingPool(nodes=4)
  >>>
  >>> def func(g,h,i):
  ...   return g+h+i
  ... 
  >>> p.map(func, [1,2,3],[4,5,6],[7,8,9])
  [12, 15, 18]
  >>>
  >>> # also can pickle stuff like lambdas 
  >>> result = pool.map(lambda x: x**2, range(10))
  >>> result
  [0, 1, 4, 9, 16, 25, 36, 49, 64, 81]
  >>>
  >>> # also does asynchronous map
  >>> result = pool.amap(pow, [1,2,3], [4,5,6])
  >>> result.get()
  [1, 32, 729]
  >>>
  >>> # or can return a map iterator
  >>> result = pool.imap(pow, [1,2,3], [4,5,6])
  >>> result
  
  >>> list(result)
  [1, 32, 729]