Data augmentation in test/validation set?

Question

It is common practice to augment data (add samples programmatically, such as random crops, etc. in the case of a dataset consisting of images) on both training and test set, or

Answer 1

I would argue that, in some cases, using data augmentation for the validation set can be helpful.

For example, I train a lot of CNNs for medical image segmentation. Many of the augmentation transforms that I use are meant to reduce the image quality so that the network is trained to be robust against such data. If the training set looks bad and the validation set looks nice, it will be hard to compare the losses during training and therefore assessing overfit will be complicated.

I would never use augmentation for the test set unless I'm using test-time augmentation to improve results or estimate aleatoric uncertainty.