Finding a file in a Python module distribution

I've written a Python package that includes a bsddb database of pre-computed values for one of the more time-consuming computations. For simplicity, my setup script installs the database file in the same directory as the code which accesses the database (on Unix, something like /usr/lib/python2.5/site-packages/mypackage/).

How do I store the final location of the database file so my code can access it? Right now, I'm using a hack based on the __file__ variable in the module which accesses the database:

dbname = os.path.join(os.path.dirname(__file__), "database.dat")

It works, but it seems... hackish. Is there a better way to do this? I'd like to have the setup script just grab the final installation location from the distutils module and stuff it into a "dbconfig.py" file that gets installed alongside the code that accesses the database.

Aaron Maenpaa

Try using pkg_resources, which is part of setuptools (and available on all of the pythons I have access to right now):

>>> import pkg_resources
>>> pkg_resources.resource_filename(__name__, "foo.config")
'foo.config'
>>> pkg_resources.resource_filename('tempfile', "foo.config")
'/usr/lib/python2.4/foo.config'

There's more discussion about using pkg_resources to get resources on the eggs page and the pkg_resources page.

Also note, where possible it's probably advisable to use pkg_resources.resource_stream or pkg_resources.resource_string because if the package is part of an egg, resource_filename will copy the file to a temporary directory.

Use pkgutil.get_data. It’s the cousin of pkg_resources.resource_stream, but in the standard library, and should work with flat filesystem installs as well as zipped packages and other importers.

That's probably the way to do it, without resorting to something more advanced like using setuptools to install the files where they belong.

Notice there's a problem with that approach, because on OSes with real a security framework (UNIXes, etc.) the user running your script might not have the rights to access the DB in the system directory where it gets installed.

Use the standard Python-3.7 library's importlib.resources module, which is more efficient than setuptools:pkg_resources (on previous Python versions, use the backported importlib_resources library).

Attention: For this to work, the folder where the data-file resides must be a regular python-package. That means you must add an __init__.py file into it, if not already there.

Then you can access it like this:

try:
  import importlib.resources as importlib_resources
except ImportError:
  # In PY<3.7 fall-back to backported `importlib_resources`.
  import importlib_resources


## Note that the actual package could have been used, 
#  not just its (string) name, with something like: 
#      from XXX import YYY as data_pkg
data_pkg = '.'
fname = 'database.dat'

db_bytes = importlib_resources.read_binary(data_pkg, fname)
# or if a file-like stream is needed:
with importlib_resources.open_binary(data_pkg, fname) as db_file:
    ...

来源：https://stackoverflow.com/questions/39104/finding-a-file-in-a-python-module-distribution

标签

python

distutils