According to the documentation, if DEBUG is set to False and something is provided under the ADMINS setting, Django will send an email
Another thing worth noting here is that settings handler500 might bypass the mechanism that sends errors on a 500 if the response from the view doesn't have a status code of 500.
If you have a handler500 set, then in that view respond with something like this.
t = loader.get_template('500.html')
response = HttpResponseServerError(
t.render(RequestContext(request, {'custom_context_var':
'IT BROKE OMG FIRE EVERYONE'})))
response.status_code = 500
return response