Removing data from a HDF5 file

Question

I\'m having a HDF5 file with one-dimensional (N x 1) dataset of compound elements - actually it\'s a time series. The data is first collected offline into the HFD5 file, and

Answer 1

Copying the data or using h5repack as you have described are the two usual ways of 'shrinking' the data in an HDF5 file, unfortunately.

The problem, as you may have guessed, is that an HDF5 file has a complicated internal structure (the file format is here, for anyone who is curious), so deleting and shrinking things just leaves holes in an identical-sized file. Recent versions of the HDF5 library can track the freed space and re-use it, but your use case doesn't seem to be able to take advantage of that.

As the other answer has mentioned, you might be able to use external links or the virtual dataset feature to construct HDF5 files that were more amenable to the sort of manipulation you would be doing, but I suspect that you'll still be copying a lot of data and this would definitely add additional complexity and file management overhead.

H5Gunlink() has been deprecated, by the way. H5Ldelete() is the preferred replacement.