Django manytomany field shows all existing objects of that type in the admin

Question

The title covers most of it. I have a model Game with a manytomany field called users this field has blank=True because it can be empty. Whenever I open this object in the a