Average weights in keras models

Question

How to average weights in Keras models, when I train few models with the same architecture with different initialisations?

Now my code looks something like this?

Answer 1

So let's assume that models is a collection of your models. First - collect all weights:

weights = [model.get_weights() for model in models]

Now - create a new averaged weights:

new_weights = list()

for weights_list_tuple in zip(*weights):
    new_weights.append(
        [numpy.array(weights_).mean(axis=0)\
            for weights_ in zip(*weights_list_tuple)])

And what is left is to set these weights in a new model:

new_model.set_weights(new_weights)

Of course - averaging weights might be a bad idea, but in case you try - you should follow this approach.