PyTorch Model's gradients are converging to zero

Question

I\'m currently working on a personal implementation of the Transformer architecture. The code I\'ve written as here.

The problem that I\'m facing is that I believe my