neural-network

问题 I'm trying to implement gradient calculation for neural networks using backpropagation. I cannot get it to work with cross entropy error and rectified linear unit (ReLU) as activation. I managed to get my implementation working for squared error with sigmoid, tanh and ReLU activation functions. Cross entropy (CE) error with sigmoid activation gradient is computed correctly. However, when I change activation to ReLU - it fails. (I'm skipping tanh for CE as it retuls values in (-1,1) range.) Is

问题 I have a problem with specifying the learning rate using the caret package with the method "mlpWeightDecay" from RSNNS package. The tuning parameters of "mlpWeightDecay" are size and decay. An example leaving size constant at 4 and tuning decay over c(0,0.0001, 0.001, 0.002): data(iris) TrainData <- iris[,1:4] TrainClasses <- iris[,5] fit1 <- train(TrainData, TrainClasses, method = "mlpWeightDecay", preProcess = c("center", "scale"), tuneGrid=expand.grid(.size = 4, .decay = c(0,0.0001, 0.001,

问题 Assuming I fit the following neural network for a binary classification problem: model = Sequential() model.add(Dense(21, input_dim=19, init='uniform', activation='relu')) model.add(Dense(80, init='uniform', activation='relu')) model.add(Dense(80, init='uniform', activation='relu')) model.add(Dense(1, init='uniform', activation='sigmoid')) # Compile model model.compile(loss='binary_crossentropy', optimizer='adam', metrics=['accuracy']) # Fit the model model.fit(x2, training_target, nb_epoch

问题 I have asked a few questions about neural networks on this website in the past and have gotten great answers, but I am still struggling to implement one for myself. This is quite a long question, but I am hoping that it will serve as a guide for other people creating their own basic neural networks in MATLAB, so it should be worth it. What I have done so far could be completely wrong. I am following the online stanford machine learning course by Professor Andrew Y. Ng and have tried to

问题 I need to write a program where part of the TensorFlow nodes need to keep being there storing some global information(mainly variables and summaries) while the other part need to be changed/reorganized as program runs. The way I do now is to reconstruct the whole graph in every iteration. But then, I have to store and load those information manually from/to checkpoint files or numpy arrays in every iteration, which makes my code really messy and error prone. I wonder if there is a way to

问题 I realize that this is probably a very niche question, but has anyone had experience with working with continuous neural networks? I'm specifically interested in what a continuous neural network may be useful for vs what you normally use discrete neural networks for. For clarity I will clear up what I mean by continuous neural network as I suppose it can be interpreted to mean different things. I do not mean that the activation function is continuous. Rather I allude to the idea of a

问题 We need to decide between Support Vector Machines and Fast Artificial Neural Network for some text processing project. It includes Contextual Spelling Correction and then tagging the text to certain phrases and their synonyms. Which will be the right approach? Or is there an alternate to both of these... Something more appropriate than FANN as well as SVM? 回答1: I think you'll get a competitive results from both of the algorithms, so you should aggregate the results... think about ensemble

问题 I wrote a vanilla autoencoder using only Dense layer. Below is my code: iLayer = Input ((784,)) layer1 = Dense(128, activation='relu' ) (iLayer) layer2 = Dense(64, activation='relu') (layer1) layer3 = Dense(28, activation ='relu') (layer2) layer4 = Dense(64, activation='relu') (layer3) layer5 = Dense(128, activation='relu' ) (layer4) layer6 = Dense(784, activation='softmax' ) (layer5) model = Model (iLayer, layer6) model.compile(loss='binary_crossentropy', optimizer='adam') (trainX, trainY),

问题 While going through the example of a tiny 2-layer neural network I noticed the result that I cannot explain. Imagine we have the following dataset with the corresponding labels: [0,1] -> [0] [0,1] -> [0] [1,0] -> [1] [1,0] -> [1] Let's create a tiny 2-layer NN which will learn to predict the outcome of a two number sequence where each number can be 0 or 1. We shall train this NN given our dataset mentioned above. import numpy as np # compute sigmoid nonlinearity def sigmoid(x): output = 1 /

问题 Drop-Out is regularization techniques. And I want to apply it to notMNIST data to reduce over-fitting to finish my Udacity Deep Learning Course Assignment.I have read the docs of tensorflow on how to call the tf.nn.dropout . And here is my code # before proceeding further. from __future__ import print_function import numpy as np import tensorflow as tf from six.moves import cPickle as pickle pickle_file = 'notMNIST.pickle' with open(pickle_file, 'rb') as f: save = pickle.load(f) train_dataset