Pruning in Keras

Question

I\'m trying to design a neural network using Keras with priority on prediction performance, and I cannot get sufficiently high accuracy by further reducing the number of lay

Answer 1

See this dedicated tooling for tf.keras. https://www.tensorflow.org/model_optimization/guide/pruning

As the overview suggests, support for latency improvements is a work in progress

Edit: Keras -> tf.keras based on LucG's suggestion.

Answer 2

If you set an individual weight to zero won't that prevent it from being updated during back propagation? Shouldn't thatv weight remain zero from one epoch to the next? That's why you set the initial weights to nonzero values before training. If you want to "remove" an entire node, just set all of the weights on that node's output to zero and that will prevent that nodes from having any affect on the output throughout training.

Answer 3

Take a look at Keras Surgeon: https://github.com/BenWhetton/keras-surgeon

I have not tried it myself, but the documentation claims that it has functions to remove or insert nodes.

Also, after looking at some papers on pruning, it seems that many researchers create a new model with less channels (or less layers), and then copy the weights from the original model to the new model.

Answer 4

Not a dedicated way :(

There's currently no easy (dedicated) way of doing this with Keras.

A discussion is ongoing at https://groups.google.com/forum/#!topic/keras-users/oEecCWayJrM.

You may also be interested in this paper: https://arxiv.org/pdf/1608.04493v1.pdf.