uWSGI works as process but not as daemon

Question

For my current flask deployment, I had to set up a uwsgi server. This is how I have created the uwsgi daemon:

sudo vim /etc/init/uwsgi.conf

Answer 1

I gave up, there was no uwsgi.log generated, and nginx just kept complaining about :

2014/03/06 01:06:28 [error] 23175#0: *22 upstream prematurely closed connection while reading response header from upstream, client: client.IP, server: my.server.IP, request: "GET / HTTP/1.1", upstream: "uwsgi://unix:/var/web/the_gelatospot/uwsgi.sock:", host: "host.ip"

for every single request. This only happened if running uwsgi as a service, as a process it started fine under any user. So this would work from command line (page responds):

$exec /var/web/the_gelatospot/mez_server.sh

This didn't (/etc/init/site_service.conf):

description "mez sites virtualenv and uwsgi_django" start on runlevel
[2345] stop on runlevel [06] respawn respawn limit 10 5 exec
/var/web/the_gelatospot/mez_server.sh

The process would start but on each request nginx would complain about the closed connection. Strangely I have this same config working just fine for 2 other apps using the same nginx version and same uwsgi version as well as both apps are mezzanine CMS apps. I tried everything I could think of and what was suggested. In the end I switched to gunicorn which works fine:

#!/bin/bash

NAME="the_gelatospot"                                          # Name of the application
DJANGODIR=/var/web/the_gelatospot              # Django project directory
SOCKFILE=/var/web/the_gelatospot/gunicorn.sock     # we will communicte using this unix socket
USER=ec2-user
GROUP=ec2-user                                             # the user to run as, the group to run as
NUM_WORKERS=3                                     # how many worker processes should Gunicorn spawn
DJANGO_SETTINGS_MODULE=settings
#DJANGO_SETTINGS_MODULE=the_gelatospot.settings             # which settings file should Django use
#DJANGO_WSGI_MODULE=the_gelatospot.wsgi                   # WSGI module name
DJANGO_WSGI_MODULE=wsgi

echo "Starting $NAME as `the_gelatospot`"

# Activate the virtual environment
cd $DJANGODIR
source ../mez_env/bin/activate
export DJANGO_SETTINGS_MODULE=$DJANGO_SETTINGS_MODULE
export PYTHONPATH=$DJANGODIR:$PYTHONPATH

# Create the run directory if it doesn't exist
RUNDIR=$(dirname $SOCKFILE)
test -d $RUNDIR || mkdir -p $RUNDIR
cd ..
# Start your Django GUnicorn
# Programs meant to be run under supervisor should not daemonize themselves (do not use --daemon)
exec gunicorn -k eventlet ${DJANGO_WSGI_MODULE}:application \
  --name $NAME \
  --workers $NUM_WORKERS \
  --log-level=debug \
  --bind=unix:$SOCKFILE

And here is the one that wouldn't work as a service (nginx complaining of prematurely closed connection and no app log data coming through).

#!/bin/bash

DJANGODIR=/var/web/the_gelatospot/                # Django project directory
cd $DJANGODIR
source ../mez_env/bin/activate
uwsgi --ini uwsgi.ini

And the uwsgi.ini:

[uwsgi]
uid = 222
gid = 500
socket = /var/web/the_gelatospot/uwsgi.sock
virtualenv = /var/web/mez_env
chdir = /var/web/the_gelatospot/
wsgi-file = /var/web/the_gelatospot/wsgi.py
pythonpath = ..
env = DJANGO_SETTINGS_MODULE=the_gelatospot.settings
die-on-term = true
master = true
chmod-socket = 666
;experiment using uwsgitop
worker = 1
;gevent = 100
processes = 1
daemonize = /var/log/nginx/uwsgi.log
logto = /var/log/nginx/uwsgi.logi
log-maxsize = 10000000
enable-threads = true

I went from gunicorn to uWSGI last year and I had no issues with it till now, it also seemed a bit faster than gunicorn. Right now I'm thinking of sticking to gunicorn. Its getting better, puts up much nicer numbers with eventlet installed, and it is easier to configure.

Hope this workaround helps. I would still like to know the issue with uWSGI and nginx but I'm stumped.

Update: So using gunicorn allowed me to run the server as a service and while toying in mezzanine I ran into this error: FileSystemEncodingChanged

To fix this I found the solution here: https://groups.google.com/forum/#!msg/mezzanine-users/bdln_Y99zQw/9HrhNSKFyZsJ

And I had to modify it a bit since I don't use supervisord, I only use upstart and a shell script. I added this to right before I execute gunicorn in my mez_server.sh file:

export LANG=en_US.UTF-8, LC_ALL=en_US.UTF-8, LC_LANG=en_US.UTF-8
exec gunicorn -k eventlet ${DJANGO_WSGI_MODULE}:application \
  --name $NAME \
  --workers $NUM_WORKERS \
  --log-level=debug \
  --bind=unix:$SOCKFILE

I also tried this fix using uWSGI like so ( a lot shorter since using uwsgi.ini ):

export LANG=en_US.UTF-8, LC_ALL=en_US.UTF-8, LC_LANG=en_US.UTF-8
exec uwsgi --ini uwsgi.ini

I am still sticking to gunicorn since it still worked with the problem and lead me in the right direction to resolve it. I was very disappointed that uWSGI provided no output in the log file even with these params, I only seen the server start process and that's it:

daemonize = /var/log/nginx/uwsgi.log
logto = /var/log/nginx/uwsgi.logi

While nginx kept throwing the disconnect error, uWSGI sat there like nothing is happening.