How to append to .mat file using scipy.io.savemat?

Question

so when I use the savemat command it tends to overwrite the file. Is there a possible way to append instead of overwriting? I know a work-around would be to put everything i

Answer 1

According to savemat docs:

file_name : str or file-like object

So you can open the file in append mode, and write, e.g.

io.savemat('temp.mat',{'data':np.ones(10)})  # write
with open('temp.mat','ab') as f:
    io.savemat(f, {'newdata':np.arange(5)})   # append
print io.loadmat('temp.mat').keys()           # read
# dict_keys(['data', '__globals__', 'newdata', '__header__', '__version__'])

Or the full description:

{'data': array([[ 1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.,  1.]]),
 '__globals__': [],
 'newdata': array([[0, 1, 2, 3, 4]]),
 '__header__': b'MATLAB 5.0 MAT-file Platform: posix, Created on: Fri Mar 13 14:14:33 2015',
 '__version__': '1.0'}

A note in https://github.com/scipy/scipy/blob/master/scipy/io/matlab/mio5.py#L34 suggests that there is a problem with appending when there's a function in the data file, but this indicates that appending isn't a problem if we are just saving arrays. But maybe further search of the scipy issues is in order.

Answer 2

Another option if you need to append huge data is the next:

self.upload1 = sio.loadmat(self.namedir,self.params)
sio.savemat('params_lo.mat', self.upload1)

With this form you load the data and add the new variables. Be careful because if they have the same name it doesn't change its values. So in order to correct this I delete all the variables that I want to change.

    self.upload = sio.loadmat(self.namedir)
    for i in self.str_vars:
       try:
          del self.upload[i]
       except:
          continue
    sio.savemat('params_lo.mat', self.upload)
    self.upload1 = sio.loadmat(self.namedir,self.params)
    sio.savemat('params_lo.mat', self.upload1)

Answer 3

It only works for one append!If you append twice, Matlab gives an error "File might be currupt".

scipy.io.savemat('temp.mat',{'data':np.ones(10)})  # write
with open('temp.mat','ab') as f:
    scipy.io.savemat(f, {'newdata1':np.arange(5)})   # append
with open('temp.mat','ab') as f:
   scipy.io.savemat(f, {'newdata2':np.arange(5)})   # append