问题
The question on overwrite array using h5py did not solve my problem. I want to edit the array values of a VGG16 model.
f = h5py.File('C:/Users/yash/.keras/models/vgg16_weights_tf_dim_ordering_tf_kernels_2.h5', mode = 'a')
ab = list(h5py.AttributeManager.keys(f))
print(list(f.attrs.keys()))
print(ab)
The code above returns:
['layer_names']
['block1_conv1', 'block1_conv2', 'block1_pool', 'block2_conv1', 'block2_conv2', 'block2_pool', 'block3_conv1', 'block3_conv2', 'block3_conv3',
'block3_pool', 'block4_conv1', 'block4_conv2', 'block4_conv3', 'block4_pool',
'block5_conv1', 'block5_conv2', 'block5_conv3', 'block5_pool', 'fc1', 'fc2',
'flatten', 'predictions']
After using this code:
print(f.attrs['layer_names'])
I get the following:
[b'block1_conv1' b'block1_conv2' b'block1_pool' b'block2_conv1'
b'block2_conv2' b'block2_pool' b'block3_conv1' b'block3_conv2'
b'block3_conv3' b'block3_pool' b'block4_conv1' b'block4_conv2'
b'block4_conv3' b'block4_pool' b'block5_conv1' b'block5_conv2'
b'block5_conv3' b'block5_pool' b'flatten' b'fc1' b'fc2' b'predictions']
How can i change the values that are contained within the f.attrs['layer_names']? I am not able to edit them mainly because using:
print(f.attrs['layer_names/block1_conv1']) returns an error.
There is a weight and bias matrix inside every block(n)_conv(n).
I want to change those values.
I am doing this in python 3, and no documentation helped me in editing these values. Mostly because i am unable to access these without using this code:
layer = h5py.AttributeManager.get(f, key = str(layerstringlist[i]))
nplayer = np.asarray(list(layer))
layerstringlist is a list of this manner:
['block1_conv1/block1_conv1_W_1:0', 'block1_conv1/block1_conv1_b_1:0', .....
'predictions/predictions_W_1:0', 'predictions/predictions_b_1:0']
This returns it correctly, but i am unable to save the modified h5 file because i do not know how to reference it in python 3.
Thanks in advance!
回答1:
I haven't seen the use of AttributeManager before, perhaps because the documentation discourages its use, http://docs.h5py.org/en/latest/high/attr.html#reference
With a file left over from other SO tests I get:
In [480]: list(h5py.AttributeManager.keys(f))
Out[480]: ['agroup', 'agroup1', 'agroup2', 'arr']
In [481]: list(f.attrs.keys())
Out[481]: []
In [482]: list(f.keys())
Out[482]: ['agroup', 'agroup1', 'agroup2', 'arr']
In this case I have not assigned any attributes to the file, hence f.attrs.keys() is empty. It appears that your file has one attribute, 'layer_names'. Its value is a list of names, which you list with print(f.attrs['layer_names']).
The AttributeManager lists the groups and datasets, not the attrs. I get the same list with f.keys().
You should be access one of these groups or datasets with:
f['block1_conv1']
If this is a group you need to index down another layer. If it is a dataset, read and write to it as described in http://docs.h5py.org/en/latest/high/dataset.html#reading-writing-data
I don't think the f.attrs['layer_names'] list is of any use to you, since it has the same information as `list(f.keys()).
Based on your comment, f['block1_conv1'] is a group, with contains several datasets. These are equivalent ways of indexing a set:
f['block1_conv1/block1_conv1_W_1:0']
f['block1_conv1']['block1_conv1_W_1:0']
In my test file
In [483]: f['arr']
Out[483]: <HDF5 dataset "arr": shape (3,), type "|V31">
I can load the dataset into memory as an array with value or [:]:
In [485]: f['arr'].value
Out[485]:
array([(123, 1, 1, 1, 1, 1, 1, 1), ( 1, 1, 1, 1, 1, 1, 1, 1),
( 1, 1, 1, 1, 1, 1, 1, 1)],
dtype=[('Status', '<u8'), ('Segments', '<u4'), ('Characterized', '<u4'), ('More_Segments', '<u4'), ('ID', '<i4'), ('Releases', '<u2'), ('Type', 'u1'), ('Track', '<i4')])
In [486]: f['arr'][:]
Out[486]:
array([(123, 1, 1, 1, 1, 1, 1, 1), ( 1, 1, 1, 1, 1, 1, 1, 1),
( 1, 1, 1, 1, 1, 1, 1, 1)],
dtype=[('Status', '<u8'), ('Segments', '<u4'), ('Characterized', '<u4'), ('More_Segments', '<u4'), ('ID', '<i4'), ('Releases', '<u2'), ('Type', 'u1'), ('Track', '<i4')])
(sorry, this example is a complicated structured array.)
I can modify the values of this dataset just as I would modify an array of the same type and shape
In [487]: f['arr']['Status']
Out[487]: array([123, 1, 1], dtype=uint64)
In [488]: f['arr']['Status'] = [1,2,3]
I cannot replace it. f['arr'] = np.arange(10) gives me an error (name already exists). f['arr'][:] = np.arange(10) gives a different error (about incompatible shapes).
I could create a new dataset with a different name
In [492]: f.create_dataset('newarray', np.arange(10))
Out[492]: <HDF5 dataset "newarray": shape (0, 1, 2, 3, 4, 5, 6, 7, 8, 9), type "<f4">
In [493]: list(f.keys())
Out[493]: ['agroup', 'agroup1', 'agroup2', 'arr', 'newarray']
I can delete a dataset with:
In [494]: del f['newarray']
In [495]: list(f.keys())
Out[495]: ['agroup', 'agroup1', 'agroup2', 'arr']
and define a new one with same name with:
In [500]: f.create_dataset('newarray', data=np.ones((3,4)))
Out[500]: <HDF5 dataset "newarray": shape (3, 4), type "<f8">
来源：https://stackoverflow.com/questions/46121210/how-to-edit-h5-files-with-h5py