deep-learning

问题 I recently downloaded fasttext pretrained model for english. I got two files: wiki.en.vec wiki.en.bin I am not sure what is the difference between the two files? 回答1: The .vec files contain only the aggregated word vectors, in plain-text. The .bin files in addition contain the model parameters, and crucially, the vectors for all the n-grams. So if you want to encode words you did not train with using those n-grams (FastText's famous "subword information"), you need to find an API that can

问题 I'm trying to stack 2 layers of tf.nn.conv2d_transpose() to up-sample a tensor. It works fine during feed forward, but I get an error during backward propagation: ValueError: Incompatible shapes for broadcasting: (8, 256, 256, 24) and (8, 100, 100, 24) . Basically, I've just set the output of the first conv2d_transpose as the input of the second one: convt_1 = tf.nn.conv2d_transpose(...) convt_2 = tf.nn.conv2d_transpose(conv_1) Using just one conv2d_transpose , everything works fine. The

问题 I'm trying to stack 2 layers of tf.nn.conv2d_transpose() to up-sample a tensor. It works fine during feed forward, but I get an error during backward propagation: ValueError: Incompatible shapes for broadcasting: (8, 256, 256, 24) and (8, 100, 100, 24) . Basically, I've just set the output of the first conv2d_transpose as the input of the second one: convt_1 = tf.nn.conv2d_transpose(...) convt_2 = tf.nn.conv2d_transpose(conv_1) Using just one conv2d_transpose , everything works fine. The

问题 Would you please guide me how to interpret the following results? 1) loss < validation_loss 2) loss > validation_loss It seems that the training loss always should be less than validation loss. But, both of these cases happen when training a model. 回答1: Really a fundamental question in machine learning. If validation loss >> training loss you can call it overfitting. If validation loss > training loss you can call it some overfitting. If validation loss < training loss you can call it some

问题 Would you please guide me how to interpret the following results? 1) loss < validation_loss 2) loss > validation_loss It seems that the training loss always should be less than validation loss. But, both of these cases happen when training a model. 回答1: Really a fundamental question in machine learning. If validation loss >> training loss you can call it overfitting. If validation loss > training loss you can call it some overfitting. If validation loss < training loss you can call it some

问题 I am trying to comprehend inner workings of the gradient accumulation in PyTorch . My question is somewhat related to these two: Why do we need to call zero_grad() in PyTorch? Why do we need to explicitly call zero_grad()? Comments to the accepted answer to the second question suggest that accumulated gradients can be used if a minibatch is too large to perform a gradient update in a single forward pass, and thus has to be split into multiple sub-batches. Consider the following toy example:

问题 I am trying to comprehend inner workings of the gradient accumulation in PyTorch . My question is somewhat related to these two: Why do we need to call zero_grad() in PyTorch? Why do we need to explicitly call zero_grad()? Comments to the accepted answer to the second question suggest that accumulated gradients can be used if a minibatch is too large to perform a gradient update in a single forward pass, and thus has to be split into multiple sub-batches. Consider the following toy example:

问题 I'm setting up the new Tensorflow Object Detection API to find small objects in large areas of satellite imagery. It works quite well - it finds all 10 objects I want, but I also get 50-100 false positives [things that look a little like the target object, but aren't]. I'm using the sample config from the 'pets' tutorial, to fine-tune the faster_rcnn_resnet101_coco model they offer. I've started small, with only 100 training examples of my objects (just 1 class). 50 examples in my validation

问题 I'm setting up the new Tensorflow Object Detection API to find small objects in large areas of satellite imagery. It works quite well - it finds all 10 objects I want, but I also get 50-100 false positives [things that look a little like the target object, but aren't]. I'm using the sample config from the 'pets' tutorial, to fine-tune the faster_rcnn_resnet101_coco model they offer. I've started small, with only 100 training examples of my objects (just 1 class). 50 examples in my validation

问题 I am building a deconvolution network. I would like to add a layer to it which is the reverse of a softmax. I tried to write a basic python function that returns the inverse of a softmax for a given matrix and put that in a tensorflow Lambda and add it to my model. I have no error but when I doing a predict I only have 0 at the exit. When I don't add this layer to my network I have output something other than zeros. This therefore justifies that they are due to my inv_softmax function which