Average weights in keras models

前端 未结 2 2052
春和景丽
春和景丽 2021-01-04 18:48

How to average weights in Keras models, when I train few models with the same architecture with different initialisations?

Now my code looks something like this?

2条回答
  •  感动是毒
    2021-01-04 19:25

    I can't comment on the accepted answer, but to make it work on tensorflow 2.0 with tf.keras I had to make the list in the loop into a numpy array:

    new_weights = list()
    for weights_list_tuple in zip(*weights): 
        new_weights.append(
            np.array([np.array(w).mean(axis=0) for w in zip(*weights_list_tuple)])
        )
    

    If different input models need to be weighted differently, np.array(w).mean(axis=0) needs to be replaced with np.average(np.array(w),axis=0, weights=relative_weights) where relative_weights is an array with a weight factor for each model.

提交回复
热议问题