pytorch

问题 I have both 3.6 and 3.8 python versions installed in my drive and I am using Anaconda for virtual environment with Python 3.6. When I say in cmd prompt: >python -V Python 3.6.10 :: Anaconda, Inc. However when I look for pip versions it shows >pip3 -V pip 20.1.1 from c:\python38\lib\site-packages\pip (python 3.8) I'm thinking this the cause of the problem when I use pip install -e . while my virtual environment is active: ERROR: Could not find a version that satisfies the requirement torch==1

问题 I found this line of code in the higher library: self.param_groups = _copy.deepcopy(other.param_groups) and I don't understand why that's needed. If anything I think it's harmful as I've outline here. You can go to the issue to see my reasons but the gist is this: Wouldn't having that deep copy mean the (outer loop) optimizer would be computing the gradients with respect to parameters no present in the computation graph? Since: the parameters of the differentiable/inner optimizer are a deep

问题 The following question is not a duplicate of How to apply layer-wise learning rate in Pytorch? because this question aims at freezing a subset of a tensor from training rather than the entire layer. I am trying out a PyTorch implementation of Lottery Ticket Hypothesis. For that, I want to freeze the weights in a model that are zero. Is the following a correct way to implement it? for name, p in model.named_parameters(): if 'weight' in name: tensor = p.data.cpu().numpy() grad_tensor = p.grad

问题 The following question is not a duplicate of How to apply layer-wise learning rate in Pytorch? because this question aims at freezing a subset of a tensor from training rather than the entire layer. I am trying out a PyTorch implementation of Lottery Ticket Hypothesis. For that, I want to freeze the weights in a model that are zero. Is the following a correct way to implement it? for name, p in model.named_parameters(): if 'weight' in name: tensor = p.data.cpu().numpy() grad_tensor = p.grad

问题 I have two pytorch tensors: X with shape (A, B, C, D) I with shape (A, B) Values in I are integers in range [0, C) . What is the most efficient way to get tensor Y with shape (A, B, D) , such that: Y[i][j][k] = X[i][j][ I[i][j] ][k] 回答1: You probably want to use torch.gather for the indexing and expand to adjust I to the required size: eI = I[..., None, None].expand(-1, -1, 1, X.size(3)) # make eI the same for the last dimension Y = torch.gather(X, dim=2, index=eI).squeeze() testing the code:

问题 I have two pytorch tensors: X with shape (A, B, C, D) I with shape (A, B) Values in I are integers in range [0, C) . What is the most efficient way to get tensor Y with shape (A, B, D) , such that: Y[i][j][k] = X[i][j][ I[i][j] ][k] 回答1: You probably want to use torch.gather for the indexing and expand to adjust I to the required size: eI = I[..., None, None].expand(-1, -1, 1, X.size(3)) # make eI the same for the last dimension Y = torch.gather(X, dim=2, index=eI).squeeze() testing the code:

问题 def save_checkpoint(state, is_best, filename='checkpoint.pth.tar'): torch.save(state, filename) if is_best: shutil.copyfile(filename, 'model_best.pth.tar') save_checkpoint({ 'epoch': epoch + 1, 'arch': args.arch, 'state_dict': model.state_dict(), 'best_prec1': best_prec1, 'optimizer': optimizer.state_dict() }, is_best) I am saving my model like this. How can I load back the model so that I can use it in other places, like cnn visualization? This is how I am loading the model now: torch.load(

问题 I'm trying to run this PyTorch code: for i, (input, target) in enumerate(train_loader): input = input.float().cuda(async=True) target = target.cuda(async=True) input_var = torch.autograd.Variable(input) target_var = torch.autograd.Variable(target) output = model(input_var) But when I try I am getting this error message: input = input.float().cuda(async=True) ^ SyntaxError: invalid syntax Process finished with exit code 1 What am I doing wrong? I already installed cuda. 回答1: Your code does not

问题 I'm trying to run this PyTorch code: for i, (input, target) in enumerate(train_loader): input = input.float().cuda(async=True) target = target.cuda(async=True) input_var = torch.autograd.Variable(input) target_var = torch.autograd.Variable(target) output = model(input_var) But when I try I am getting this error message: input = input.float().cuda(async=True) ^ SyntaxError: invalid syntax Process finished with exit code 1 What am I doing wrong? I already installed cuda. 回答1: Your code does not

问题 I was wondering how are layer weights and biases initialized by default? E.g. if I create the linear layer torch.nn.Linear(5,100) How are weights and biases for this layer initialized by default? 回答1: PyTorch 1.0 Most layers are initialized using Kaiming Uniform method. Example layers include Linear, Conv2d, RNN etc. If you are using other layers, you should look up that layer on this doc. If it says weights are initialized using U(...) then its Kaiming Uniform method. Bias is initialized