hdf5

问题 I'm on pandas 0.14.1. Assume I need to index data by two timestamps in a hierarchical index using timezones. When saving the resulted DataFrame to hdf5 I seem to lose timezone-awareness: import pandas as pd dti1 = pd.DatetimeIndex(start=pd.Timestamp('20000101'), end=pd.Timestamp('20000102'), freq='D', tz='EST5EDT') dti2 = pd.DatetimeIndex(start=pd.Timestamp('20000102'), end=pd.Timestamp('20000103'), freq='D', tz='EST5EDT') mux = pd.MultiIndex.from_arrays([dti1, dti2]) df = pd.DataFrame(0,

问题 I'm new in PyTables and I want to retrieve a dataset from a HDF5 using natural naming but I'm getting this error using this input: f = tables.open_file("filename.h5", "r") f.root.group-1.dataset-1.read() group / does not have a child named group and if I try: f.root.group\-1.dataset\-1.read() group / does not have a child named group unexpected character after line continuation character I can't change names in the groups because is big data from an experiment. 回答1: You can't use the minus

问题 Using Numpy and h5py, it is possible to create ‘compound datatype’ datasets to be stored in an hdf5-file: import h5py import numpy as np # # Create a new file using default properties. # file = h5py.File('compound.h5','w') # # Create a dataset under the Root group. # comp_type = np.dtype([('fieldA', 'i4'), ('fieldB', 'f4')]) dataset = file.create_dataset("comp", (4,), comp_type) It is also possible to use various compression filters in a ‘compression pipeline’, among them the ‘scale-offset’

问题 I want to use the HDF5 libraries in my C++ program. I am using the VS 2010 x64 compiler and CMake 3.8.0rc2 on Windows 7. The HDF5 version I installed is 1.8.10 (installed by running the official "Installer"). In my CMakeLists file, I added the following lines: FIND_PACKAGE ( HDF5 REQUIRED ) INCLUDE_DIRECTORIES (${HDF5_INCLUDE_DIRS}) SET (HDF5_LIBS ${HDF5_LIBS} ${HDF5_LIBRARIES}) ... target_link_libraries(${PROJECT_NAME} ${Boost_LIBRARIES} ${HDF5_LIBS}) CMake shows the following error message

问题 I am trying to load a mat file for the Street View House Numbers (SVHN) Dataset http://ufldl.stanford.edu/housenumbers/ in Python with the following code import h5py labels_file = './sv/train/digitStruct.mat' f = h5py.File(labels_file) struct= f.values() names = struct[1].values() print(names[1][1].value) I get [<HDF5 object reference>] but I need to know the actual string 回答1: To get an idea of the data layout you could execute h5dump ./sv/train/digitStruct.mat but there are also other

问题 I am trying to write regular python objects (which several key/value pairs) to a hdf5 file. I am using h5py 2.7.0 with python 3.5.2.3. Right now, I am trying to write one object in its entirety to a dataset: #...read dataset, store one data object in 'obj' #obj could be something like: {'value1': 0.09, 'state': {'angle_rad': 0.034903, 'value2': 0.83322}, 'value3': 0.3} dataset = h5File.create_dataset('grp2/ds3', data=obj) This produces an error as the underlying dtype can not be converted to

问题 I have some sample code that generates a 3d Numpy array -- I am then saving this data into a h5py file using h5 file. How can I then "append" the second dataset along the 4th dimension? Or, how can I write another 3d dataset along the 4th dimension (or new axis) of an existing .h5 file? I have read documentation that I could find, and none of the examples seem to address this. My code is shown below: import h5py import numpy as np dataset1 = np.random.rand(240,240,250); dataset2 = np.random

问题 I have a sampling application that acquires 250,000 samples per second, buffers them in memory and eventually appends to an HDFStore provided by pandas . In general, this is great. However, I have a thread that runs and continually empties the data acquisition device ( DAQ ) and it needs to run on a somewhat regular basis. A deviation of about a second tends to break things. Below is an extreme case of the timings observed. Start indicates a DAQ read starting, Finish is when it finishes, and

问题 Checkpoint snippet: checkpointer = ModelCheckpoint(filepath=os.path.join(savedir, "mid/weights.{epoch:02d}.hd5"), monitor='val_loss', verbose=1, save_best_only=False, save_weights_only=False) hist = model.fit_generator( gen.generate(batch_size = batch_size, nb_classes=nb_classes), samples_per_epoch=593920, nb_epoch=nb_epoch, verbose=1, callbacks=[checkpointer], validation_data = gen.vld_generate(VLD_PATH, batch_size = 64, nb_classes=nb_classes), nb_val_samples=10000 ) I trained my model on a

问题 This is my first go at trying to create an HDF5 file from scratch using the Low-Level commands via MATLAB. My issue is that I am having a hard time trying to write data to each specific member in the datatype on my dataset. First, I create a new HDF5 file, and set the right layer of groups: new_h5 = H5F.create('new_hdf5_file.h5','H5F_ACC_TRUNC','H5P_DEFAULT','H5P_DEFAULT'); new_h5 = H5G.create(new_h5,'first','H5P_DEFAULT','H5P_DEFAULT','H5P_DEFAULT'); new_h5 = H5G.create(new_h5,'second','H5P