keras: how to save the training history attribute of the history object

Question

In Keras, we can return the output of model.fit to a history as follows:

 history = model.fit(X_train, y_train, 
                     batch_size         


        

Answer 1

The easiest way:

Saving:

np.save('my_history.npy',history.history)

Loading:

history=np.load('my_history.npy',allow_pickle='TRUE').item()

Then history is a dictionary and you can retrieve all desirable values using the keys.

Answer 2

An other way to do this:

As history.history is a dict, you can convert it as well to a pandas DataFrame object, which can then be saved to suit your needs.

Step by step:

import pandas as pd

# assuming you stored your model.fit results in a 'history' variable:
history = model.fit(x_train, y_train, epochs=10)

# convert the history.history dict to a pandas DataFrame:     
hist_df = pd.DataFrame(history.history) 

# save to json:  
hist_json_file = 'history.json' 
with open(hist_json_file, mode='w') as f:
    hist_df.to_json(f)

# or save to csv: 
hist_csv_file = 'history.csv'
with open(hist_csv_file, mode='w') as f:
    hist_df.to_csv(f)

Answer 3

A history objects has a history field is a dictionary which helds different training metrics spanned across every training epoch. So e.g. history.history['loss'][99] will return a loss of your model in a 100th epoch of training. In order to save that you could pickle this dictionary or simple save different lists from this dictionary to appropriate file.

Answer 4

The model history can be saved into a file as follows

import json
hist = model.fit(X_train, y_train, epochs=5, batch_size=batch_size,validation_split=0.1)
with open('file.json', 'w') as f:
    json.dump(hist.history, f)

Answer 5

What I use is the following:

    with open('/trainHistoryDict', 'wb') as file_pi:
        pickle.dump(history.history, file_pi)

In this way I save the history as a dictionary in case I want to plot the loss or accuracy later on.

Answer 6

The above answers are useful when saving history at the end of the training process. If you want to save the history during the training, the CSVLogger callback will be helpful.

Below code saves the model weight and history training in form of a datasheet file log.csv.

model_cb = tf.keras.callbacks.ModelCheckpoint(filepath=checkpoint_path)
history_cb = tf.keras.callbacks.CSVLogger('./log.csv', separator=",", append=False)

history = model.fit(callbacks=[model_cb, history_cb])