Python multicore programming [duplicate]

Question

This question already has an answer here:

• Threading in Python [closed] 7 answers

Please consider a class as follow:

class Foo:
    def __init__(self, data):
        self.data = data

    def do_task(self):
        #do something with data 

In my application I've a list containing several instances of Foo class. The aim is to execute do_task for all Foo objects. A first implementation is simply:

 #execute tasks of all Foo Object instantiated
 for f_obj in my_foo_obj_list:
     f_obj.do_task()

I'd like to take advantage of multi-core architecture sharing the for cycle between 4 CPUs of my machine.

What's the best way to do it?

Answer 1

You can use process pools in multiprocessing module.

def work(foo):
    foo.do_task()

from multiprocessing import Pool

pool = Pool()
pool.map(work, my_foo_obj_list)
pool.close()
pool.join()

Answer 2

Instead of going through all the multithreading/multicore basics, I would like to reference a Post by Ryan W. Smith: Multi-Core and Distributed Programming in Python

He will go into details how you can utilize multiple cores and use those concepts. But please be careful with that stuff if you are not familiar with general multithreading concepts.

Functional Programming will also allow you to customize the algorithm/function for each core.