multiprocessing - sharing a complex object

Question

I\'ve got a large dict-like object that needs to be shared between a number of worker processes. Each worker reads a random subset of the information in the obj

Answer 1

You're confusing the object instance numeri, with its Manager listaNumeri. This can be illustrated by making a few minor modifications to the code:

First add aget_list_idmethod to class numeri(object) which returns the id of the actual internal data structure being used:

    ...                                                   
    def get_list_id(self):  # added method
        return id(self.nl)

Then modifyproduce()to use it:

def produce(listaNumeri):
    print 'producing', id(listaNumeri)
    print ' with list_id', listaNumeri.get_list_id()  # added
    return id(listaNumeri)

Lastly, be sure to expose the new method as a part of the numManager interface:

def main():
    numManager.register('numeri', numeri, exposed=['getLen', 'appendi',
                                                   'svuota', 'stampa',
                                                   'get_list_id'])  # added
    ...                                                   

Afterwards you'll see something like the following output:

13195568
------------ Process
producing 12739600
 with list_id 13607080
producing 12739600
 with list_id 13607080
producing 12739600
 with list_id 13607080
producing 12739600
 with list_id 13607080
producing 12739600
 with list_id 13607080
--------------- Pool
producing 13690384
 with list_id 13607080
producing 13691920
 with list_id 13607080
producing 13691888
 with list_id 13607080
producing 13691856
 with list_id 13607080
producing 13691824
 with list_id 13607080

As this shows, even though there's a different Manager object for each Pool process, they're all using (sharing) the same "managed" data object.