h5py OSError: Unable to open file (File signature not found)

问题:

I'm a bit confused about an error I'm receiving when using h5py. I'm trying to apply a python script to loop through sets of h5py files located in different directories. For example, the first set of h5py files is located at

Reduced/rho=0.75/2/Data/snapshots

When I run the python script from a sub directory of Reduced

Reduced/test_h5py

with the following python script

import h5py import numpy as np import matplotlib.pyplot as plt import matplotlib.colors as colors                import cmocean import os  de.logging_setup.rootlogger.setLevel('ERROR')   # Plot writes                                                                                                                            path = '../rho=0.75/2/Data/snapshots' for filename in os.listdir(path):     with h5py.File(path+'/'+filename,'r') as file: 

everything works fine, the script loops through the data and gives me an output. Now, the issue arises when I try to apply the same python script to other data i.e at the set of h5py files located at

Reduced/rho=0.75/4/Data/snapshots

Now, when I run the previous python script with just a modification in the path from 2 to 4

import h5py import numpy as np import matplotlib.pyplot as plt import matplotlib.colors as colors                import cmocean import os  de.logging_setup.rootlogger.setLevel('ERROR')   # Plot writes                                                                                                                            path = '../rho=0.75/4/Data/snapshots' for filename in os.listdir(path):     with h5py.File(path+'/'+filename,'r') as file: 

I get the following error

Traceback (most recent call last):   File "newest_edit.py", line 17, in <module>     with h5py.File(path+'/'+filename,'r') as file:   File "/usr/local/lib/python3.5/site-packages/h5py/_hl/files.py", line 269, in __init__     fid = make_fid(name, mode, userblock_size, fapl, swmr=swmr)   File "/usr/local/lib/python3.5/site-packages/h5py/_hl/files.py", line 99, in make_fid     fid = h5f.open(name, flags, fapl=fapl)   File "h5py/_objects.pyx", line 54, in h5py._objects.with_phil.wrapper   File "h5py/_objects.pyx", line 55, in h5py._objects.with_phil.wrapper   File "h5py/h5f.pyx", line 78, in h5py.h5f.open OSError: Unable to open file (file signature not found) 

Can anyone make sense of this? I thought it might be a os.listdir() error but after searching I didn't find anything to solve the issue. Thanks for your help.

EDIT:

I forgot to mention. When I run the python script in a Jupyter notebook and apply the script to individual h5py files it works just fine for both the 2 and 4 paths. The h5py files open and I can retrieve the data stored on them, so I can't imagine it would be corrupted files or not in hdf5 format.

回答1:

I don't really know what I'm talking about here so sorry in advance if this doesn't make sense or help or just isn't related.

It's an extremely annoying error that no one has attempted to answer in over a year so I thought the tiny bit of knowledge I have might help.

I had a similar problem whilst working on a cluster. It turned out that h5py uses something called flock() and that this can sometimes have a negative effect on the filesystem performance.

Because of this flock() is sometimes not allowed. If you are working on someone elses filesystem then ask admin if they will alow it and if not then I can't find a way around it (I have not tried to find other libraries that don't use flock()).

If it is your own filesystem then you should be able to allow flock() but I'm not sure how to do this. This may help you if it's NFS

I would certainly be interested to hear how you got on?

文章来源: h5py OSError: Unable to open file (File signature not found)