neural-network

问题 I have a minimal example of a neural network with a back-propagation trainer, testing it on the IRIS data set. I started of with 7 hidden nodes and it worked well. I lowered the number of nodes in the hidden layer to 1 (expecting it to fail), but was surprised to see that the accuracy went up. I set up the experiment in azure ml, just to validate that it wasn't my code. Same thing there, 98.3333% accuracy with a single hidden node. Can anyone explain to me what is happening here? 回答1: First,

问题 Is there any implementation (or straightforward description) of a training algorithm for feed-forward neural networks which doesn't use a sigmoid or linear squash-function, but a non-differentiable one, such as the heaviside-function? I already found a paper on such an algorithm, but no according implementation, which I find bewildering, as it seems to me, there should be something out. Any hints? 回答1: Backpropagation will not work with the heavyside function because its derivate is zero in

问题 I know when we are using convolution layers in a neural net we usually use padding and mainly constant padding(e.g. zero padding). And there are different kinds of padding(e.g. symmetric, reflective, constant). But I am not sure what are the advantages and disadvantages of using different padding methods and when to use which one. 回答1: it really depends on the situation for what the neural network is intended. I would not tell it pros and cons. This time the world cannot put into a binary

问题 I'm new to Caffe. I am trying to implement a Fully Convolution Neural Network (FCN-8s) for semantic segmentation. I have image data and label data, which are both images. This is for pixel-wise predictions. I tried using ImageData as the data type, but it asks for an integer label, which is not applicable to this scenario. Kindly advise as how to I can give Caffe a 2D label. Should I prefer LMDB instead of ImageData? If so, how do I proceed? I could not find any good tutorial/documentation

问题 I know when we are using convolution layers in a neural net we usually use padding and mainly constant padding(e.g. zero padding). And there are different kinds of padding(e.g. symmetric, reflective, constant). But I am not sure what are the advantages and disadvantages of using different padding methods and when to use which one. 回答1: it really depends on the situation for what the neural network is intended. I would not tell it pros and cons. This time the world cannot put into a binary

问题 I am encountering In place operation error when I am trying to index a leaf variable to update gradients with customized Shrink function. I cannot work around it. Any help is highly appreciated! import torch.nn as nn import torch import numpy as np from torch.autograd import Variable, Function # hyper parameters batch_size = 100 # batch size of images ld = 0.2 # sparse penalty lr = 0.1 # learning rate x = Variable(torch.from_numpy(np.random.normal(0,1,(batch_size,10,10))), requires_grad=False

问题 I'm trying to define a triplet loss using descriptor from a CNN's output, but this error showed up when I try to train the network. My definition of loss function: def compute_loss(descriptor, margin): diff_pos = descriptor[0:1800:3] - descriptor[1:1800:3] diff_neg = descriptor[0:1800:3] - descriptor[2:1800:3] Ltriplet = np.maximum(0, 1 - tf.square(diff_neg)/(tf.square(diff_pos) + margin)) Lpair = tf.square(diff_pos) Loss = Ltriplet + Lpair return Loss here descriptor is the outcome of CNN,

问题 I am attempting to construct a neural network incorporating convolution and LSTM (using the Torch library) to be trained by Q-learning or Advantage-learning, both of which require propagating state T+1 through the network before updating the weights for state T. Having to do an extra propagation would cut performance and that's bad, but not too bad; However, the problem is that there is all kinds of state bound up in this. First of all, the Torch implementation of backpropagation has some

问题 how to flatten input inside the nn.Sequential Model = nn.Sequential(x.view(x.shape[0],-1), nn.Linear(784,256), nn.ReLU(), nn.Linear(256,128), nn.ReLU(), nn.Linear(128,64), nn.ReLU(), nn.Linear(64,10), nn.LogSoftmax(dim=1)) 回答1: You can create a new module/class as below and use it in the sequential as you are using other modules (call Flatten() ). class Flatten(torch.nn.Module): def forward(self, x): batch_size = x.shape[0] return x.view(batch_size, -1) Ref: https://discuss.pytorch.org/t

问题 I currently study the Neural Networks theory and I see that everywhere it is written that it consists of the following layers: Input Layer Hidden Layer(s) Output Layer I see some graphical descriptions that show the input layer as real nodes in the net, while others show this layer as just a vector of values [x1, x2, ... xn] What is the correct structure? Is the "input layer" a real layer of neurons? Or is this just abstractly named as layer, while it really is just the input vector? Adding