No module named deploy when trying to start pyramid app with mod_wsgi

Question

trying to use mod_wsgi for the first time for my pyramid app, but I keep getting an ImportError: No module named deploy when I try to access the site

in my /etc/apache2/sites-available/domain.com

<VirtualHost *:80>  
    ServerName domain.com
    ServerAlias www.domain.com
    ServerAdmin admin@domain.com
    DocumentRoot /data/app
    ErrorLog /data/app/apache_error.log

    WSGIApplicationGroup %{GLOBAL}
    WSGIPassAuthorization On
    WSGIDaemonProcess pyramid user=www-data group=www-data \
        processes=2 threads=4 \
        python-path=/data/app/lib/python2.6/site-packages/
    WSGIScriptAlias / /data/app/pyramid.wsgi

    <Directory /data/app>
        WSGIProcessGroup pyramid
        Order allow,deny
        Allow from all
    </Directory>
</VirtualHost>

in /data/app/pyramid.wsgi

venv = '/data/app/bin/activate_this.py'
execfile(venv, dict(__file__=venv))

ALLDIRS = ['/data/app/lib/python2.6/site-packages']

import sys, site, os

site.addsitedir('/data/app/lib/python2.6/site-packages')

sys.path.append('/data/app/app')
os.environ['PYTHON_EGG_CACHE'] = '/data/app/python-eggs'

from pyramid.paster import get_app, setup_logging
ini_path = '/data/app/app/development.ini'
setup_logging(ini_path)
application = get_app(ini_path, 'main')

when I try to access domain.com in my browser, I get the The server encountered an internal error or misconfiguration and was unable to complete your request error. Here's the print out of my error log

[Mon Apr 22 20:43:13 2013] [error] test
[Mon Apr 22 20:43:13 2013] [error] mod_wsgi (pid=6795): Target WSGI script '/data/app/pyramid.wsgi' cannot be loaded as Python module.
[Mon Apr 22 20:43:13 2013] [error] mod_wsgi (pid=6795): Exception occurred processing WSGI script '/data/app/pyramid.wsgi'.
[Mon Apr 22 20:43:13 2013] [error] Traceback (most recent call last):
[Mon Apr 22 20:43:13 2013] [error]   File "/data/app/pyramid.wsgi", line 30, in <module>
[Mon Apr 22 20:43:13 2013] [error]     from pyramid.paster import get_app, setup_logging
[Mon Apr 22 20:43:13 2013] [error]   File "/data/app/lib/python2.6/site-packages/pyramid-1.4-py2.6.egg/pyramid/paster.py", line 3, in <module>
[Mon Apr 22 20:43:13 2013] [error]     from paste.deploy import (
[Mon Apr 22 20:43:13 2013] [error] ImportError: No module named deploy

Note that the print of test on the first line is from my manually-added print 'test' at the top of /data/app/lib/python2.6/site-packages/pyramid-1.4-py2.6.egg/pyramid/paster.py to make sure it is loading from the file that I expected...

now if I start the app from pserve

$ /bin/pserve app/development.ini --reload

the app is started successfully, console log below

test
Starting subprocess with file monitor
test
Starting server in PID 9132.
serving on http://0.0.0.0:6543

I also tried going into python directly to import paster.py, no problem as well

$ cd /data/app/lib/python2.6/site-packages/pyramid-1.4-py2.6.egg/pyramid/
$ /data/app/bin/python
Python 2.6.5 (r265:79063, Oct  1 2012, 22:04:36)
[GCC 4.4.3] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> import pyramid.paster as p
test
>>> print p.__file__
/data/app/lib/python2.6/site-packages/pyramid-1.4-py2.6.egg/pyramid/paster.pyc
>>>

permissions should be fine as well

$ ls /data/app -l
-rw-rw-r-- 1 root www-data 4148 2013-04-22 21:06 apache_error.log
drwxrwsr-x 4 root www-data 4096 2013-04-22 12:05 app
drwxrwsr-x 2 root www-data 4096 2013-04-22 12:04 bin
drwxrwsr-x 2 root www-data 4096 2013-04-22 11:58 include
drwxrwsr-x 3 root www-data 4096 2013-04-22 11:58 lib
-rwxrwxr-x 1 root www-data  893 2013-04-22 12:23 pyramid.wsgi
drwxrwsr-x 2 root www-data 4096 2013-04-22 12:07 python-eggs

what am I missing? thanks!

Edit www-data is the correct user as well

$ ps aux | grep apache2
root      2599  0.0  1.4 185200 14552 ?        Ss   10:00   0:02 /usr/sbin/apache2 -k start
www-data  9064  0.0  0.8 185664  8940 ?        Sl   21:06   0:00 /usr/sbin/apache2 -k start
www-data  9065  0.0  0.8 185664  8940 ?        Sl   21:06   0:00 /usr/sbin/apache2 -k start
www-data  9095  0.0  1.1 187292 11388 ?        S    21:06   0:00 /usr/sbin/apache2 -k start
www-data  9096  0.0  1.1 187292 11388 ?        S    21:06   0:00 /usr/sbin/apache2 -k start
www-data  9097  0.0  1.0 186768 10472 ?        S    21:06   0:00 /usr/sbin/apache2 -k start
www-data  9098  0.0  1.0 186768 10472 ?        S    21:06   0:00 /usr/sbin/apache2 -k start
www-data  9099  0.0  1.0 186768 10472 ?        S    21:06   0:00 /usr/sbin/apache2 -k start
root      9189  0.0  0.0   7624   912 pts/0    S+   21:42   0:00 grep apache2

Edit per Graham's suggestion, i replaced print 'test' in the paste.py with

import paste as p
print p.__path__

the printout when accessing the site through apache was /usr/local/lib/python2.6/dist-packages/paste, not the virtualenv. I have added all the examples that I could fine online to point to my virtualenv in pyramid.wsgi as well as the pythonpath in the apache config file, but clearly I am still not getting it right. Where else do I try?

Answer 1

Solved it finally. From the console log it's obvious that my main python installation in /usr/local/lib has a paste module installed that does not have deploy handler defined.

RedBaron's comment to set WSGIPythonHome worked, but the problem is I have multiple apps so I cannot just point WSGIPythonHome globally to a specific app's virtualenv.

Anyhow, I went through mod_wsgi docs on virtualenv again (https://code.google.com/p/modwsgi/wiki/VirtualEnvironments#Baseline_Environment) and saw that WSGIPythonHome should be pointing to a "virgin environment". literally

$ virtualenv --no-site-packages BASELINE

and then pointing WSGIPythonHome

WSGIPythonHome /usr/local/pythonenv/BASELINE

and point WSGIDaemonProcess to my app's specific virtualenv (which I already did at the beginning)

LESSON LEARNT: Pay more attention to the docs...

---

Finally, my pyramid.wsgi can be greatly simplified to only the following

import os
os.environ['PYTHON_EGG_CACHE'] = '/data/app/python-eggs'

from pyramid.paster import get_app, setup_logging
ini_path = '/data/app/app/development.ini'
setup_logging(ini_path)
application = get_app(ini_path, 'main')

Answer 2

In my case reinstalling PasteDeploy in the virtual environment fixed the problem, not sure why though.

Answer 3

The error would indicate that the PasteDeploy package is not installed.

• https://pypi.python.org/pypi/PasteDeploy