PyTorch multiprocessing: are ACER implementations moving gradients correctly?

Question

I am implementing the ACER architecture, which performs asynchronous parameter update as A3C. Looking at existing PyTorch implementations, I often see the parameter update p