Numpy ImportError when deploying Flask App using mod_wsgi/Apache2

Question

I'm trying to get a Flask app running on AWS EC2 (standard Ubuntu AMI) through an Apache2 Webserver. My app internally uses Numpy. I have tested the following:

1. Without Numpy, the App runs through Apache2. With Numpy, it trips on the import numpy and throws a 500 Server error (See logs below).

2. With Numpy, the app runs fine when invoked directly from the command line (i.e. python app.py). The numpy bits work as expected and I can query app endpoints externally.

I also printed the system path (print sys.path) just before the import numpy as np and made sure that the appropriate site-packages directory is in the path. I also manually specified the WSGIPythonPath to include the site-packages directory. I tried setting the number of threads to 1 and editing the apache2.conf file. None of these efforts have been successful.

Below is my directory structure and contents of the relevant files.

Root folder /var/www/html/webserver_mockup

Root folder contents

/var/www/html/webserver_mockup/app.wsgi /var/www/html/webserver_mockup/mockup/__init__.py /var/www/html/webserver_mockup/mockup/app.py

app.wsgi

(Edited according to Graham's comment below)

#!/usr/bin/python
import sys
import site
site.addsitedir('/home/ubuntu/.local/lib/python2.7/site-packages')
sys.path.insert(0, "/var/www/html/webserver_mockup")

from mockup.app import app as application    

/etc/apache2/sites-available/000-default.conf

(Edited according to Graham's comment below)

WSGIPythonPath /usr/local/lib/python2.7/site-packages/

<VirtualHost *:80>
    ServerAdmin webmaster@localhost
    DocumentRoot /var/www/html

    WSGIDaemonProcess webserver_mockup threads=5
    WSGIScriptAlias / /var/www/html/webserver_mockup/app.wsgi

    <Directory />
        WSGIProcessGroup %{GLOBAL}
        WSGIApplicationGroup %{GLOBAL}
        Order allow,deny
        Allow from all
    </Directory>
    LogLevel warn
    ErrorLog ${APACHE_LOG_DIR}/error.log
    CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>

Apache error log

[Fri Apr 07 04:09:41.062282 2017] [mpm_event:notice] [pid 7221:tid 
140325391972224] AH00489: Apache/2.4.18 (Ubuntu) mod_wsgi/4.3.0 
Python/2.7.12 configured -- resuming normal operations
[Fri Apr 07 04:09:41.062358 2017] [core:notice] [pid 7221:tid 
140325391972224] AH00094: Command line: '/usr/sbin/apache2'
[Fri Apr 07 04:09:46.770159 2017] [wsgi:error] [pid 7225:tid 
140325250004736] [client 24.6.50.183:58332] mod_wsgi (pid=7225): Target 
WSGI script '/var/www/html/webserver_mockup/app.wsgi' cannot 
be loaded as Python module.
[Fri Apr 07 04:09:46.770300 2017] [wsgi:error] [pid 7225:tid 
140325250004736] [client 24.6.50.183:58332] mod_wsgi (pid=7225): 
Exception occurred processing WSGI script 
'/var/www/html/webserver_mockup/app.wsgi'.
[Fri Apr 07 04:09:46.770370 2017] [wsgi:error] [pid 7225:tid 
 140325250004736] [client 24.6.50.183:58332] Traceback (most recent 
call last):
[Fri Apr 07 04:09:46.770432 2017] [wsgi:error] [pid 7225:tid 
140325250004736] [client 24.6.50.183:58332]   File 
"/var/www/html/webserver_mockup/app.wsgi", line 7, in <module>
[Fri Apr 07 04:09:46.770534 2017] [wsgi:error] [pid 7225:tid 
140325250004736] [client 24.6.50.183:58332]     from mockup.app import 
app as application
[Fri Apr 07 04:09:46.770612 2017] [wsgi:error] [pid 7225:tid 
140325250004736] [client 24.6.50.183:58332]   File 
"/var/www/html/webserver_mockup/mockup/app.py", line 12, in 
<module>
[Fri Apr 07 04:09:46.770693 2017] [wsgi:error] [pid 7225:tid 
140325250004736] [client 24.6.50.183:58332]     import numpy as np
[Fri Apr 07 04:09:46.770755 2017] [wsgi:error] [pid 7225:tid 
140325250004736] [client 24.6.50.183:58332] ImportError: No module 
named numpy

My question is: Why is Apache2 not finding Numpy despite the appropriate site-packages being in its path?

Answer 1

This:

site.addsitedir('/home/ubuntu/.local/lib/python2.7/site-packages/numpy')

should have been:

site.addsitedir('/home/ubuntu/.local/lib/python2.7/site-packages')

But the whole way you are setting up Python virtual environments is not best practice anyway. Suggest you read:

• http://modwsgi.readthedocs.io/en/develop/user-guides/virtual-environments.html

Also try and switch to daemon mode, it is better to use daemon mode than embedded mode.

Answer 2

numpy via pip is not installed at /home/ubuntu/.local/lib/python2.7/site-packages but it should be at /home/ubuntu/.local/lib64/python2.7/site-packages or something like that.

Just try add another site.addsitedir in your app.wsgi to that folder should work. Something like this :

site.addsitedir('/home/ubuntu/.local/lib64/python2.7/site-packages')