According to the documentation, if DEBUG is set to False and something is provided under the ADMINS setting, Django will send an email
DEBUG
False
ADMINS
Make sure you have DEBUG = False