hdf5

问题 I'm using HDF5DotNet in C# and I have a very large array (several GB) that I want to write to an HDF5 file. It's too big to store the whole thing in memory, so I'm generating regions of it at a time and want to write them out, but still have it look like one big array when it's read back out. I know this is possible with HDF5 but the documentation for the .NET API is somewhat sparse. I wrote some short example code with a 5 x 3 array filled with values 1..15: const int ROWS = 5; const int

问题 I am using the follows 1) VS 2010 C++ 2) Debug Win 32 3) The library from here http://www.hdfgroup.org/HDF5/release/obtain5.html Basically I downloaded Windows (32-bit) Compilers: CMake VS 2010 C, C++, IVF 12, RWDI and installed it. I tried to include a sample code in my C++ application and ran into the following ***HDF5 library version mismatched error*** The HDF5 header files used to compile this application do not match the version used by the HDF5 library to which this application is

问题 I am using the follows 1) VS 2010 C++ 2) Debug Win 32 3) The library from here http://www.hdfgroup.org/HDF5/release/obtain5.html Basically I downloaded Windows (32-bit) Compilers: CMake VS 2010 C, C++, IVF 12, RWDI and installed it. I tried to include a sample code in my C++ application and ran into the following ***HDF5 library version mismatched error*** The HDF5 header files used to compile this application do not match the version used by the HDF5 library to which this application is

问题 I am new to hdf5 and was trying to store a DataFrame row into the hdf5 format. I was to append a row at different locations within the file; however, every time I append it shows up at an array in a single column rather than a single value in multiple columns. I have tried both h5py and pandas and it seems like pandas is the better option for appending. Additionally, I have really been trying a lot of different methods. Truly, any help would be greatly appreciated. Here is me sending an array

问题 I have a very large dataset I write to hdf5 in chunks via append like so: with pd.HDFStore(self.train_store_path) as train_store: for filepath in tqdm(filepaths): with open(filepath, 'rb') as file: frame = pickle.load(file) if frame.empty: os.remove(filepath) continue try: train_store.append( key='dataset', value=frame, min_itemsize=itemsize_dict) os.remove(filepath) except KeyError as e: print(e) except ValueError as e: print(frame) print(e) except Exception as e: print(e) The data is far

问题 I'm using C++ & HDF5 to write a file. But run into problems with it. This is the code I use: void fileRead::writeFile(string name, const vector<double>* data) { int dimn = data->size(); hsize_t dim[1] = {data->size()}; //-> 2^13!!! hid_t sid = H5Pcreate(H5P_DATASET_CREATE); hid_t didProp = H5Screate_simple(1,dim,NULL); H5Pset_layout(sid, H5D_COMPACT); hid_t did = H5Dcreate(fid, name.c_str(),H5T_IEEE_F64LE, didProp, H5P_DEFAULT, sid,H5P_DEFAULT); H5Dwrite (did, H5T_NATIVE_DOUBLE, H5S_ALL, H5S

问题 I am installing Python 2.7 in addition to 2.7. When installing PyTables again for 2.7, I get this error - Found numpy 1.5.1 package installed. .. ERROR:: Could not find a local HDF5 installation. You may need to explicitly state where your local HDF5 headers and library can be found by setting the HDF5_DIR environment variable or by using the --hdf5 command-line option. I am not clear on the HDF installation. I downloaded again - and copied it into a /usr/local/hdf5 directory. And tried to

问题 I am installing Python 2.7 in addition to 2.7. When installing PyTables again for 2.7, I get this error - Found numpy 1.5.1 package installed. .. ERROR:: Could not find a local HDF5 installation. You may need to explicitly state where your local HDF5 headers and library can be found by setting the HDF5_DIR environment variable or by using the --hdf5 command-line option. I am not clear on the HDF installation. I downloaded again - and copied it into a /usr/local/hdf5 directory. And tried to

问题 I have some data that I share between Python and Matlab. I used to do it by saving NumPy arrays in MATLAB-style .mat files but would like to switch to HDF5 datasets. However, I've noticed a funny feature: when I save a NumPy array in an HDF5 file (using h5py) and then read it in Matlab (using h5read), it ends up being transposed. Is there something I'm missing? Python code: import numpy as np import h5py mystuff = np.random.rand(10,30) f = h5py.File('/home/user/test.h5', 'w') f['mydataset'] =

问题 I have a class like this: class C: def __init__(self, id, user_id, photo): self.id = id self.user_id = user_id self.photo = photo I need to create millions of these objects. id is an integer as well as user_id but photo is a bool array of size 64. My boss wants me to store all of them inside hdf5 files. I also need to be able to make queries according to their user_id attributes to get all of the photos that have the same user_id. Firstly, how do I store them? Or even can I? And secondly,