Python 2.7.2 shelve fails on OSX

Question

Using the shelve standard lib on Python 2.7.2, I wrote an extremely simple test to create a persistent data file and then immediately open it for printing:

import os
import shelve

shelf_filename = str(__file__.split('.')[0] + '.dat')

#delete the shelf file if it exists already.
try:
    os.remove(shelf_filename)
    print "DELETED LEFTOVER SHELF FILE", shelf_filename
except OSError:
    pass

#create a new shelf, write some data, and flush it to disk
shelf_handle = shelve.open(shelf_filename)
print "OPENED", shelf_filename, "WITH SHELF"
shelf_handle['foo'] = 'bar'
shelf_handle.close()
print "FLUSHED AND CLOSED THE SHELF"

#re-open the shelf we just wrote, read/print the data, and close it
shelf_handle = shelve.open(shelf_filename)
print "RE-OPENED", shelf_filename, "WITH SHELF"
print 'foo:', shelf_handle.get('foo')
shelf_handle.close()

#delete the shelf file
os.remove(shelf_filename)

However this script fails unexpectedly when it tries to re-open the shelf it just created:

DELETED LEFTOVER SHELF FILE shelve_test.dat
OPENED shelve_test.dat WITH SHELF
FLUSHED AND CLOSED THE SHELF
Traceback (most recent call last):
  File "shelve_test.py", line 21, in <module>
    shelf_handle = shelve.open(shelf_filename)
  File ".../shelve.py", line 239, in open
    return DbfilenameShelf(filename, flag, protocol, writeback)
  File ".../shelve.py", line 223, in __init__
    Shelf.__init__(self, anydbm.open(filename, flag), protocol, writeback)
  File ".../anydbm.py", line 82, in open
    raise error, "db type could not be determined"
anydbm.error: db type could not be determined

This is about the most basic possible usage of shelve so I really don't understand what I'm missing, and AFAIK I'm following the docs to the letter. Can somebody with a little shelve experience please tell me what's going on?

UPDATE: This script apparently works in certain platforms and not others, which is really kind of surprising for a Python standard lib. Definitely confirmed this does NOT work on:

Python 2.7.2 (default, May 15 2013, 13:46:05) 
[GCC 4.2.1 Compatible Apple LLVM 4.2 (clang-425.0.28)] on darwin

Answer 1

This version of Python shelve is using a deprecated dbm package, and the issue can be solved by specifying a different dbm as follows:

import anydbm
anydbm._defaultmod = __import__('dumbdbm')

If those two lines are added to the script above then everything works as expected. This really needs to be fixed in shelve.

Answer 2

shelve doesn't expect you to specify the extension on the filename. Instead it gives the file an extension based on which db was used to create the file.

So when you call shelve.open("shelve_test.dat"), you are creating a file that shelve expects to be created in using dumbdbm, but is instead created with a different (better) database manager.

Instead, you should be calling shelve.open("shelve_test"), that way shelve can choose to save the file using the correct extension. This also applies when opening an already created file.