multi-layer perceptron (MLP) architecture: criteria for choosing number of hidden layers and size of the hidden layer?

Question

If we have 10 eigenvectors then we can have 10 neural nodes in input layer.If we have 5 output classes then we can have 5 nodes in output layer.But what is the criteria for

Answer 1

To automate the selection of the best number of layers and best number of neurons for each of the layers, you can use genetic optimization.

The key pieces would be:

1. Chromosome: Vector that defines how many units in each hidden layer (e.g. [20,5,1,0,0] meaning 20 units in first hidden layer, 5 in second, ... , with layers 4 and 5 missing). You can set a limit on the maximum number number of layers to try, and the max number of units in each layer. You should also place restrictions of how the chromosomes are generated. E.g. [10, 0, 3, ... ] should not be generated, because any units after a missing layer (the '3,...') would be irrelevant and would waste evaluation cycles.
2. Fitness Function: A function that returns the reciprocal of the lowest training error in the cross-validation set of a network defined by a given chromosome. You could also include the number of total units, or computation time if you want to find the "smallest/fastest yet most accurate network".

You can also consider:

• Pruning: Start with a large network, then reduce the layers and hidden units, while keeping track of cross-validation set performance.
• Growing: Start with a very small network, then add units and layers, and again keep track of CV set performance.