deep-learning

问题 I am trying to make an attention model with Bi-LSTM using word embeddings. I came across How to add an attention mechanism in keras?, https://github.com/philipperemy/keras-attention-mechanism/blob/master/attention_lstm.py and https://github.com/keras-team/keras/issues/4962. However, I am confused about the implementation of Attention-Based Bidirectional Long Short-Term Memory Networks for Relation Classification . So, _input = Input(shape=[max_length], dtype='int32') # get the embedding layer

问题 Embedding Layers in PyTorch are listed under "Sparse Layers" with the limitation: Keep in mind that only a limited number of optimizers support sparse gradients: currently it’s optim.SGD (cuda and cpu), and optim.Adagrad (cpu) What is the reason for this? For example in Keras I can train an architecture with an Embedding Layer using any optimizer. 回答1: Upon closer inspection sparse gradients on Embeddings are optional and can be turned on or off with the sparse parameter: class torch.nn

问题 Embedding Layers in PyTorch are listed under "Sparse Layers" with the limitation: Keep in mind that only a limited number of optimizers support sparse gradients: currently it’s optim.SGD (cuda and cpu), and optim.Adagrad (cpu) What is the reason for this? For example in Keras I can train an architecture with an Embedding Layer using any optimizer. 回答1: Upon closer inspection sparse gradients on Embeddings are optional and can be turned on or off with the sparse parameter: class torch.nn

问题 I created an LSTM using Keras with TensorFlow as backend. Before a minibatch with a num_step of 96 is given to the training, the hidden state of the LSTM is set to true values of a previous time step. First the parameters and data: batch_size = 10 num_steps = 96 num_input = num_output = 2 hidden_size = 8 X_train = np.array(X_train).reshape(-1, num_steps, num_input) Y_train = np.array(Y_train).reshape(-1, num_steps, num_output) X_test = np.array(X_test).reshape(-1, num_steps, num_input) Y_test

问题 The following is how I understand the point of parameter sharing in RNNs: In regular feed-forward neural networks, every input unit is assigned an individual parameter, which means that the number of input units (features) corresponds to the number of parameters to learn. In processing e.g. image data, the number of input units is the same over all training examples (usually constant pixel size * pixel size * rgb frames). However, sequential input data like sentences can come in highly

问题 I have developed a rest API using Flask to expose a Python Keras Deep Learning model (CNN for text classification). I have a very simple script that loads the model into memory and outputs class probabilities for a given text input. The API works perfectly locally. However, when I git push heroku master , I get Compiled slug size: 588.2M is too large (max is 500M) . The model is 83MB in size, which is quite small for a Deep Learning model. Notable dependencies include Keras and its tensorflow

问题 I had followed this blog --> https://medium.com/@teyou21/training-your-object-detection-model-on-tensorflow-part-2-e9e12714bdf , and built a SSD Mobilenet model which is pre-trained on the COCO Dataset called "ssd_mobilenet_v2_quantized_coco". What happens here is that it perfectly detects my new classes, but I want to include the pre-trained classes as well. I tried changing the number of classes to 96 ( 90 pre-trained + 6 new ) and edited the "labelmap.pbtxt" with the name and corresponding

问题 I had followed this blog --> https://medium.com/@teyou21/training-your-object-detection-model-on-tensorflow-part-2-e9e12714bdf , and built a SSD Mobilenet model which is pre-trained on the COCO Dataset called "ssd_mobilenet_v2_quantized_coco". What happens here is that it perfectly detects my new classes, but I want to include the pre-trained classes as well. I tried changing the number of classes to 96 ( 90 pre-trained + 6 new ) and edited the "labelmap.pbtxt" with the name and corresponding

问题 can anyone tell me how is backpropagation done in Keras? I read that it is really easy in Torch and complex in Caffe, but I can't find anything about doing it with Keras. I am implementing my own layers in Keras (A very beginner) and would like to know how to do the backward propagation. Thank you in advance 回答1: You simply don't. (Late edit: except when you are creating custom training loops, only for advanced uses) Keras does backpropagation automatically. There's absolutely nothing you

问题 can anyone tell me how is backpropagation done in Keras? I read that it is really easy in Torch and complex in Caffe, but I can't find anything about doing it with Keras. I am implementing my own layers in Keras (A very beginner) and would like to know how to do the backward propagation. Thank you in advance 回答1: You simply don't. (Late edit: except when you are creating custom training loops, only for advanced uses) Keras does backpropagation automatically. There's absolutely nothing you