uWSGI will not run in mixed Python environment in order to operate correctly with nginx and run Django app

Question

Environment: Ubuntu 16.04 (with system Python at 2.7.12) running in Vagrant/Virtualbox on Windows 10 host

Python Setup: System python verified by doing python -V with no virtualenv's activated. Python 3.5 is also installed, and I've done pipenv --three to create the virtualenv for this project. Doing python -V within the activated virtualenv (pipenv shell to activate) shows Python 3.5.2.

Additional Background: I'm developing a Wagtail 2 app. Wagtail 2 requires Django 2 which, of course, requires Python 3. I have other Django apps on this machine that were developed in Django 1.11/Python 2.7 and are served by Apache. We are moving to Django 2/Python 3 for development going forward and are moving to nginx/uWSGI for serving the apps.

I have gone through many tutorials/many iterations. All Vagrant port mapping is set up fine with nginx serving media/static files and passing requests upstream to the Django app on a unix socket, but this is giving a 502 Gateway not found error because uWSGI will not run correctly. Therefore, right now I'm simply running the following from the command line to try to get uWSGI to run: uwsgi --ini /etc/uwsgi/sites/my_site.com.ini. This file contains:

[uwsgi]

uid = www-data
gid = www-data

plugin = python35

# Django-related settings
# the base directory (full path)
chdir=/var/sites/my_site
# Django's wsgi file
wsgi-file = my_site.wsgi
# the virtualenv (full path)
virtualenv=/root/.local/share/virtualenvs/my_site-gmmiTMID

# process-related settings
# master
master = True
# maximum number of worker processes
processes = 10

# the socket (use the full path to be safe)
socket = /tmp/my_site.sock

# clear environment on exit
vacuum = True

I've tried installing uWSGI in the following ways:

• system-wide with pip install uwsgi as well as pip3 install uwsgi
• using apt-get install uwsgi uwsgi-plugin-python3

I've ensured that only one install is in place at a time by uninstalling any previous uwsgi installs. The latter install method places uwsgi-core in usr/bin and also places in usr/bin shortcuts to uwsgi, uwsgi_python3, and uwsgi_python35.

In the .ini file I've also tried plugin = python3. I've also tried from the command line:

• uwsgi_python3 --ini /etc/uwsgi/sites/my_site.com.ini
• uwsgi_python35 --ini /etc/uwsgi/sites/my_site.com.ini

I've tried executing the uwsgi ... .ini commands from both within the activated virtual environment and with the virtualenv deactivated. Each of the three command line uwsgi ... .ini executions (uwsgi ..., uwsgi_python3 ... and uwsgi_python35 ...) DO cause the .ini file to be executed, but each time I'm getting the following error (the last two lines of the following statements):

[uwsgi] implicit plugin requested python35
[uWSGI] getting INI configuration from /etc/uwsgi/sites/my_site.com.ini
*** Starting uWSGI 2.0.12-debian (64bit) on [Wed Mar  7 03:54:44 2018] ***
compiled with version: 5.4.0 20160609 on 31 August 2017 21:02:04
os: Linux-4.4.0-116-generic #140-Ubuntu SMP Mon Feb 12 21:23:04 UTC 2018
nodename: vagrant-ubuntu-trusty-64
machine: x86_64
clock source: unix
pcre jit disabled
detected number of CPU cores: 2
current working directory: /vagrant/my_site
detected binary path: /usr/bin/uwsgi-core
setgid() to 33
setuid() to 33
chdir() to /var/sites/my_site
your processes number limit is 7743
your memory page size is 4096 bytes
detected max file descriptor number: 1024
lock engine: pthread robust mutexes
thunder lock: disabled (you can enable it with --thunder-lock)
uwsgi socket 0 bound to UNIX address /tmp/my_site.sock fd 3
Python version: 3.5.2 (default, Nov 23 2017, 16:37:01)  [GCC 5.4.0 20160609]
Set PythonHome to /root/.local/share/virtualenvs/my_site-gmmiTMID
Fatal Python error: Py_Initialize: Unable to get the locale encoding
ImportError: No module named 'encodings'

If I go into the Python command line within the activated virtualenv and do import encodings, it imports fine (no message - just comes back to command line). A search for this particular error has turned up nothing of use. Any idea why the ImportError: No module named 'encodings' is coming up?

UPDATE - PROBLEM STILL OCCURRING

I'm using pipenv, and it stores individual virtualenvs in the /home/username/.local/share/virtualenvs folder. Though I was able to start uWSGI from the command line by executing the uWSGI config file as the vagrant user (see comment below), I have still not been able start the service with /home/vagrant/.local/share/virtualenvs/my_venv in the uWSGI config file. I tried adding the vagrant user to the www-data group and adding the www-data user to the vagrant group. I also put both read and execute permission for the world on the whole path (including the individual venv), but the uWSGI service will still not start.

A HACKISH WORKAROUND

I did finally get the uWSGI service to start by copying the venv to /opt/virtualenvs/my_venv. I was then able to start the service with sudo service uwsgi start. Ownership of that whole path is root:root.

STILL LOOKING FOR A SOLUTION...

This solution is not optimal since I am now executing from a virtualenv that will have to be updated when the default virtualenv is updated since this location is not the default for pipenv, so I'm still looking for answers. Perhaps it is a Ubuntu permissions error, but I just can't find the problem.

Answer 1

It might be the problem with your virtual environment. Try the following

rm -rf /root/.local/share/virtualenvs/my_site-gmmiTMID
virtualenv -p python3 /root/.local/share/virtualenvs/my_site-gmmiTMID
source /root/.local/share/virtualenvs/my_site-gmmiTMID/bin/activate
pip install -r requirements.txt

and in uwsgi conf try to change

virtualenv=/root/.local/share/virtualenvs/my_site-gmmiTMID

to

home=/root/.local/share/virtualenvs/my_site-gmmiTMID

Reference: http://uwsgi-docs.readthedocs.io/en/latest/Options.html#virtualenv