backpropagation

XOR neural network does not learn

由 拜拜、爱过 提交于 2019-12-08 07:55:09
问题 I am trying to solve the very simple non-linear problem. It is XOR gate. I my school knowledge. XOR can be solve by using 2 input nodes, 2 hidden layer nodes. And 1 output. It is binary classification problem. I generate the 1000 of random integer number it is 0 or 1 and then do backpropagation. But for some unknown reason my network has not learned anything. The training accuracy is constant at 50 . # coding: utf-8 import matplotlib import torch import torch.nn as nn from torch.autograd

Multi-layer neural network back-propagation formula (using stochastic gradient descent)

由 一曲冷凌霜 提交于 2019-12-08 05:13:20
问题 Using the notations from Backpropagation calculus | Deep learning, chapter 4, I have this back-propagation code for a 4-layer (i.e. 2 hidden layers) neural network: def sigmoid_prime(z): return z * (1-z) # because σ'(x) = σ(x) (1 - σ(x)) def train(self, input_vector, target_vector): a = np.array(input_vector, ndmin=2).T y = np.array(target_vector, ndmin=2).T # forward A = [a] for k in range(3): a = sigmoid(np.dot(self.weights[k], a)) # zero bias here just for simplicity A.append(a) # Now A

Back propagation algorithm: error computation

由 混江龙づ霸主 提交于 2019-12-08 04:48:25
问题 I am currently writing a back propagation script. I am unsure how to go about updating my weight values. Here is an image just to make things simple. My question: How is the error calculated and applied? I do know that k1 and k2 produce error values. I know that k1 and k2 produce individual error values (target - output). I do not however know if these are to be used. Am I supposed to use the mean value of both error values and then apply that single error value to all of the weights? Or am I

How does Keras handle backpropagation in multiple outputs

由 一笑奈何 提交于 2019-12-08 04:30:48
问题 For a network architecture like this: +---+ input1--->| CNN | -------| +---+ | | +---+ +-------+ +-------+ input2--->| CNN | ----| Concat|-----|----| VGG |---- Main_out +---+ +-------+ | +-------+ | | +---+ | | input3--->| CNN | --------| Aux_out +---+ How does the backpropagation flow go? I mean, there are two backpropagation steps? Or the only one that comes from the Main_out updates the weights. I am using loss weights for each output: model.compile(loss="categorical_crossentropy"

Tune input features using backprop in keras

由 依然范特西╮ 提交于 2019-12-07 13:42:02
问题 I am trying to implement discriminant condition codes in Keras as proposed in Xue, Shaofei, et al., "Fast adaptation of deep neural network based on discriminant codes for speech recognition." The main idea is you encode each condition as an input parameter and let the network learn dependency between the condition and the feature-label mapping. On a new dataset instead of adapting the entire network you just tune these weights using backprop. For example say my network looks like this X ----

How does Keras handle backpropagation in multiple outputs

由 余生长醉 提交于 2019-12-07 01:55:30
For a network architecture like this: +---+ input1--->| CNN | -------| +---+ | | +---+ +-------+ +-------+ input2--->| CNN | ----| Concat|-----|----| VGG |---- Main_out +---+ +-------+ | +-------+ | | +---+ | | input3--->| CNN | --------| Aux_out +---+ How does the backpropagation flow go? I mean, there are two backpropagation steps? Or the only one that comes from the Main_out updates the weights. I am using loss weights for each output: model.compile(loss="categorical_crossentropy",optimizer=OPT,metrics=["accuracy"], loss_weights={'main_output': 1., 'aux_output': 0.2} The losses for

Multi-layer neural network back-propagation formula (using stochastic gradient descent)

由 北城余情 提交于 2019-12-06 21:01:28
Using the notations from Backpropagation calculus | Deep learning, chapter 4 , I have this back-propagation code for a 4-layer (i.e. 2 hidden layers) neural network: def sigmoid_prime(z): return z * (1-z) # because σ'(x) = σ(x) (1 - σ(x)) def train(self, input_vector, target_vector): a = np.array(input_vector, ndmin=2).T y = np.array(target_vector, ndmin=2).T # forward A = [a] for k in range(3): a = sigmoid(np.dot(self.weights[k], a)) # zero bias here just for simplicity A.append(a) # Now A has 4 elements: the input vector + the 3 outputs vectors # back-propagation delta = a - y for k in [2, 1

Multi-layer neural network won't predict negative values

由 时光怂恿深爱的人放手 提交于 2019-12-06 17:06:02
问题 I have implemented a multilayer perceptron to predict the sin of input vectors. The vectors consist of four -1,0,1's chosen at random and a bias set to 1. The network should predict the sin of sum of the vectors contents. eg Input = <0,1,-1,0,1> Output = Sin(0+1+(-1)+0+1) The problem I am having is that the network will never predict a negative value and many of the vectors' sin values are negative. It predicts all positive or zero outputs perfectly. I am presuming that there is a problem

XOR neural network does not learn

由 女生的网名这么多〃 提交于 2019-12-06 14:54:15
I am trying to solve the very simple non-linear problem. It is XOR gate. I my school knowledge. XOR can be solve by using 2 input nodes, 2 hidden layer nodes. And 1 output. It is binary classification problem. I generate the 1000 of random integer number it is 0 or 1 and then do backpropagation. But for some unknown reason my network has not learned anything. The training accuracy is constant at 50 . # coding: utf-8 import matplotlib import torch import torch.nn as nn from torch.autograd import Variable matplotlib.use('TkAgg') # My buggy OSX 10.13.6 requires this import matplotlib.pyplot as

How Many Epochs Should a Neural Net Need to Learn to Square? (Testing Results Included)

由 不羁的心 提交于 2019-12-06 03:25:53
Okay, let me preface this by saying that I am well aware that this depends on MANY factors, I'm looking for some general guidelines from people with experience. My goal is not to make a Neural Net that can compute squares of numbers for me, but I thought it would be a good experiment to see if I implemented the Backpropagation algorithm correctly. Does this seem like a good idea? Anyways, I am worried that I have not implemented the learning algorithm (fully) correctly. My Testing (Results): Training Data : 500 randomly generated numbers between .001 and .999 using Java's Random Network
订阅 backpropagation