According to the documentation, if DEBUG is set to False and something is provided under the ADMINS setting, Django will send an email
DEBUG
False
ADMINS
Sorry if it is too naive, but in my case the emails were sent but were going directly to the SPAM folder. Before trying more complicated things check your SPAM folder first.