backpropagation

I have programmed a Neural Network in Java and am now working on the back-propagation algorithm. I've read that batch updates of the weights will cause a more stable gradient search instead of a online weight update. As a test I've created a time series function of 100 points, such that x = [0..99] and y = f(x) . I've created a Neural Network with one input and one output and 2 hidden layers with 10 neurons for testing. What I am struggling with is the learning rate of the back-propagation algorithm when tackling this problem. I have 100 input points so when I calculate the weight change dw_

We are writing a small ANN which is supposed to categorize 7000 products into 7 classes based on 10 input variables. In order to do this we have to use k-fold cross validation but we are kind of confused. We have this excerpt from the presentation slide: What are exactly the validation and test sets? From what we understand is that we run through the 3 training sets and adjust the weights (single epoch). Then what do we do with the validation? Because from what I understand is that the test set is used to get the error of the network. What happens next is also confusing to me. When does the

Currently I'm learning about neural networks and I'm trying to create an application that can be trained to recognize handwritten characters. For this problem I use a feed-forward neural network and it seems to work when I train it to recognize 1, 2 or 3 different characters. But when I try to make the network learn more than 3 characters it will stagnate at a error percentage around the 40 - 60%. I tried with multiple layers and less/more neurons but I can't seem to get it right, now I'm wondering if a feedforward neural network is capable of recognizing that much information. Some statistics

I am computing the backpropagation algorithm for a sparse autoencoder. I have implemented it in python using numpy and in matlab . The code is almost the same, but the performance is very different. The time matlab takes to complete the task is 0.252454 seconds while numpy 0.973672151566, that is almost four times more. I will call this code several times later in a minimization problem so this difference leads to several minutes of delay between the implementations. Is this a normal behaviour? How could I improve the performance in numpy? Numpy implementation: Sparse.rho is a tuning parameter

Although both of the above methods provide better score for better closeness of prediction, still cross-entropy is preferred. Is it in every cases or there are some peculiar scenarios where we prefer cross-entropy over MSE? Cross-entropy is prefered for classification , while mean squared error is one of the best choices for regression . This comes directly from the statement of the problems itself - in classification you work with very particular set of possible output values thus MSE is badly defined (as it does not have this kind of knowledge thus penalizes errors in incompatible way). To