Setting an Item in nested dictionary with __setitem__

Question

Here's what I did, trying to create a wrapper for a dict-like database, among other functions:

    class database(object):
        def __init__(self, name):
            self.name = name
            self.db = anydbm.open(name, 'c')

        def __getitem__(self, key):
            key = str(key)
            try:
                self.db = anydbm.open(self.name, 'w') 
            except Exception,e: 
                raise e
            else:
                return cPickle.loads(self.db[key])
            finally:  
                self.db.close()

        def __setitem__(self, key, value):
            key = str(key)
            value = cPickle.dumps(value)
            try:
                self.db = anydbm.open(self.name, 'w')
            except Exception,e:
                print e 
            else:
                self.db[key] = value
            finally:  
                self.db.close()

When I try to define a new key in a dict nested within, it seems getitem is returning a value instead of a reference, so the definition is not finally modified after the setitem.

>>> from database import database
>>> db = database('test')
>>> db['a'] = {'alpha':'aaa'}
>>> db['a']['alpha'] = 'bbb'
>>> print db
{'a': {'alpha': 'aaa'}} //The value is not modified

Answer 1

Well, you have to understand what you are doing:

db['a'] = {'alpha':'aaa'}

is equivalent to

db.__setitem__('a', {'alpha':'aaa'})

so this dumps the dict to the disk. However when you do

db['a']['alpha'] = 'bbb'

you first load the dict from disk

tmp = db.__getitem__('a') # except tmp is pushed on the stack

and then change this dict:

tmp['alpha'] = 'bbb'

This obviously has no effect on the data dumped to disk, because your object is not involved anymore.

To make this work you cannot return a simple dict, but instead you need another object that tracks changes and writes them back to disk.

Btw, you're writing shelve. It has the same problem:

Because of Python semantics, a shelf cannot know when a mutable persistent-dictionary entry is modified. By default modified objects are written only when assigned to the shelf (see Example). If the optional writeback parameter is set to True, all entries accessed are also cached in memory, and written back on sync() and close(); this can make it handier to mutate mutable entries in the persistent dictionary, but, if many entries are accessed, it can consume vast amounts of memory for the cache, and it can make the close operation very slow since all accessed entries are written back (there is no way to determine which accessed entries are mutable, nor which ones were actually mutated).