How to set cache settings while using h5py high level interface?

Question

I\'m trying to increase cache size for my HDF5 files, but it doesn\'t seem to be working. This is what I have:

import h5py

with h5py.File(\"test.h5\", \'w\'         


        

Answer 1

If you are using h5py version 2.9.0 or newer, see Mike's answer.

---

According to the docs, get_access_plist() returns a copy of the file access property list. So it is not surprising that modifying the copy does not affect the original.

It appears the high-level interface does not provide a way to change the cache settings.

Here is how you could do it using the low-level interface.

propfaid = h5py.h5p.create(h5py.h5p.FILE_ACCESS)
settings = list(propfaid.get_cache())
print(settings)
# [0, 521, 1048576, 0.75]

settings[2] *= 5
propfaid.set_cache(*settings)
settings = propfaid.get_cache()
print(settings)
# (0, 521, 5242880, 0.75)

The above creates a PropFAID. We can then open the file and get a FileID this way:

import contextlib
with contextlib.closing(h5py.h5f.open(
                        filename, flags=h5py.h5f.ACC_RDWR, fapl=propfaid)) as fid:
    # 
    settings = list(fid.get_access_plist().get_cache())
    print(settings)
    # [0, 521, 5242880, 0.75]

And we can use the fid to open the file with the high-level interface by passing fid to h5py.File:

    f = h5py.File(fid)
    print(f.id.get_access_plist().get_cache())
    # (0, 521, 5242880, 0.75)

Thus, you can still use the high-level interface, but it takes some fiddling to get there. On the other hand, if you distill it to just the essentials, perhaps it isn't so bad:

import h5py
import contextlib

filename = '/tmp/foo.hdf5'
propfaid = h5py.h5p.create(h5py.h5p.FILE_ACCESS)
settings = list(propfaid.get_cache())
settings[2] *= 5
propfaid.set_cache(*settings)
with contextlib.closing(h5py.h5f.open(filename, fapl=propfaid)) as fid:
    f = h5py.File(fid)