Looping through training data in Neural Networks Backpropagation Algorithm

Question

How many times do I use a sample of training data in one training cycle? Say I have 60 training data. I go through the 1st row and do a forward pass and adjust weights using results from backward pass. Using the sigmoidal function as below:

Forward pass 
Si = sum of (Wi * Uj)
Ui = f(Si) = 1 / 1 + e^ - Si

Backward pass 
Output Cell = (expected -Ui)(f'(Si)), where 
f'(Si) = Ui(1-Ui)

Do I then go through the 2nd row and do the same process as the 1st or do I go around the 1st row until the error is less?

I hope someone can help please

Answer 1

Training the network

You should use each instance of the training set once per training epoch.

A training epoch is a complete cycle through your dataset.

After you've looped through the dataset and calculated the deltas, you should adjust the weights of the network. Then you may perform a new forward pass on the neural network and do another training epoch, looping through your training dataset.

Graphical representation
A really great graphical representation of backpropagation may be found at this link.

---

Single-step training

There are two approaches to train you network to perform classification on a dataset. The easiest method is called single-step or online learning. This is the method you will find in most litterature, and it is also the fastest to converge. As you train your network you will calculate the deltas for each layer and adjust the weights for each instance of your dataset.

Thus if you have a dataset of 60 instances, this means you should have adjusted the weights 60 times before the training epoch is over.

Batch training

The other approach is called batch training or offline learning. This approach often yields a network with a lower residual error. When you train the network you should calculate the deltas for each layer for every instance of the dataset, and then finally average the individual deltas and correct the weights once per epoch.

If you have a dataset of 60 instances, this means you should have adjusted the weights once before the training epoch is over.