Distributed evaluation of validation dataset in pytorch

Question

I\'m using some ready-made scripts for distributed training of my model, and not understanding very well the mechanics behind it. Basically, it uses torch.distributed