Use python 2 shelf in python 3

Question

I have data stored in a shelf file created with python 2.7

When I try to access the file from python 3.4, I get an error:

>>> import shelve
>         


        

Answer 1

Edited: You may need to rename your database. Read on...

Seems like pickle is not the culprit here. shelve relies also in anydbm (Python 2.x) or dbm (Python 3) to create/open a database and store the pickled information.

I created (manually) a database file using the following:

# Python 2.7
import anydbm
anydbm.open('database2', flag='c')

and

# Python 3.4
import dbm
dbm.open('database3', flag='c')

In both cases, it creates the same kind of database (may be distribution dependent, this is on Debian 7):

$ file *
database2:    Berkeley DB (Hash, version 9, native byte-order)
database3.db: Berkeley DB (Hash, version 9, native byte-order)

anydbm can open database3.db without problems, as expected:

>>> anydbm.open('database3')

Notice the lack of .db when specifying the database name, though. But dbm chokes on database2, which is weird:

>>> dbm.open('database2')
Traceback (most recent call last):
  File "", line 1, in 
  File "/usr/lib/python3.4/dbm/__init__.py", line 88, in open
    raise error[0]("db type could not be determined")
dbm.error: db type could not be determined

unless I change the name of the name of the database to database2.db:

$ mv database2 database2.db
$ python3
>>> import dbm
>>> dbm.open('database2')
<_dbm.dbm object at 0x7fa7eaefcf50>

So, I suspect a regression on the dbm module, but I haven't checked the documentation. It may be intended :-?

NB: Notice that in my case, the extension is .db, but that depends on the database being used by dbm by default! Create an empty shelf using Python 3 to figure out which one are you using and what is it expecting.