In app engine flexible environment, I'm trying to run django on postgresql db using GIS extensions. Can run locally, and it eventually finished deploying to cloud successfully. However, when run from cloud I get a "ERROR: Server Error" and the log file has the trace shown below.
Essentially, it says it cannot read files in the folder "/env/lib/python3.5/site-packages/django/contrib/gis/gdal". That folder is definitely there locally. Here's the full trace:
Traceback (most recent call last):
File "/env/lib/python3.5/site-packages/gunicorn/arbiter.py", line 578, in spawn_worker
worker.init_process()
File "/env/lib/python3.5/site-packages/gunicorn/workers/base.py", line 126, in init_process
self.load_wsgi()
File "/env/lib/python3.5/site-packages/gunicorn/workers/base.py", line 135, in load_wsgi
self.wsgi = self.app.wsgi()
File "/env/lib/python3.5/site-packages/gunicorn/app/base.py", line 67, in wsgi
self.callable = self.load()
File "/env/lib/python3.5/site-packages/gunicorn/app/wsgiapp.py", line 65, in load
return self.load_wsgiapp()
File "/env/lib/python3.5/site-packages/gunicorn/app/wsgiapp.py", line 52, in load_wsgiapp
return util.import_app(self.app_uri)
File "/env/lib/python3.5/site-packages/gunicorn/util.py", line 352, in import_app
__import__(module)
File "/home/vmagent/app/mysite/wsgi.py", line 22, in <module>
application = get_wsgi_application()
File "/env/lib/python3.5/site-packages/django/core/wsgi.py", line 13, in get_wsgi_application
django.setup(set_prefix=False)
File "/env/lib/python3.5/site-packages/django/__init__.py", line 27, in setup
apps.populate(settings.INSTALLED_APPS)
File "/env/lib/python3.5/site-packages/django/apps/registry.py", line 108, in populate
app_config.import_models()
File "/env/lib/python3.5/site-packages/django/apps/config.py", line 202, in import_models
self.models_module = import_module(models_module_name)
File "/env/lib/python3.5/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "/env/lib/python3.5/site-packages/django/contrib/auth/models.py", line 4, in <module>
from django.contrib.auth.base_user import AbstractBaseUser, BaseUserManager
File "/env/lib/python3.5/site-packages/django/contrib/auth/base_user.py", line 52, in <module>
class AbstractBaseUser(models.Model):
File "/env/lib/python3.5/site-packages/django/db/models/base.py", line 124, in __new__
new_class.add_to_class('_meta', Options(meta, app_label))
File "/env/lib/python3.5/site-packages/django/db/models/base.py", line 331, in add_to_class
value.contribute_to_class(cls, name)
File "/env/lib/python3.5/site-packages/django/db/models/options.py", line 214, in contribute_to_class
self.db_table = truncate_name(self.db_table, connection.ops.max_name_length())
File "/env/lib/python3.5/site-packages/django/db/__init__.py", line 33, in __getattr__
return getattr(connections[DEFAULT_DB_ALIAS], item)
File "/env/lib/python3.5/site-packages/django/db/utils.py", line 211, in __getitem__
backend = load_backend(db['ENGINE'])
File "/env/lib/python3.5/site-packages/django/db/utils.py", line 115, in load_backend
return import_module('%s.base' % backend_name)
File "/env/lib/python3.5/importlib/__init__.py", line 126, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
File "/env/lib/python3.5/site-packages/django/contrib/gis/db/backends/postgis/base.py", line 6, in <module>
from .features import DatabaseFeatures
File "/env/lib/python3.5/site-packages/django/contrib/gis/db/backends/postgis/features.py", line 1, in <module>
from django.contrib.gis.db.backends.base.features import BaseSpatialFeatures
File "/env/lib/python3.5/site-packages/django/contrib/gis/db/backends/base/features.py", line 4, in <module>
from django.contrib.gis.db.models import aggregates
File "/env/lib/python3.5/site-packages/django/contrib/gis/db/models/__init__.py", line 3, in <module>
from django.contrib.gis.db.models.aggregates import * # NOQA
File "/env/lib/python3.5/site-packages/django/contrib/gis/db/models/aggregates.py", line 1, in <module>
from django.contrib.gis.db.models.fields import ExtentField
File "/env/lib/python3.5/site-packages/django/contrib/gis/db/models/fields.py", line 3, in <module>
from django.contrib.gis import forms, gdal
File "/env/lib/python3.5/site-packages/django/contrib/gis/forms/__init__.py", line 3, in <module>
from .fields import ( # NOQA
File "/env/lib/python3.5/site-packages/django/contrib/gis/forms/fields.py", line 4, in <module>
from django.contrib.gis.geos import GEOSException, GEOSGeometry
File "/env/lib/python3.5/site-packages/django/contrib/gis/geos/__init__.py", line 5, in <module>
from .collections import ( # NOQA
File "/env/lib/python3.5/site-packages/django/contrib/gis/geos/collections.py", line 11, in <module>
from django.contrib.gis.geos.geometry import GEOSGeometry, LinearGeometryMixin
File "/env/lib/python3.5/site-packages/django/contrib/gis/geos/geometry.py", line 11, in <module>
from django.contrib.gis import gdal
File "/env/lib/python3.5/site-packages/django/contrib/gis/gdal/__init__.py", line 28, in <module>
from django.contrib.gis.gdal.datasource import DataSource
File "/env/lib/python3.5/site-packages/django/contrib/gis/gdal/datasource.py", line 39, in <module>
from django.contrib.gis.gdal.driver import Driver
File "/env/lib/python3.5/site-packages/django/contrib/gis/gdal/driver.py", line 5, in <module>
from django.contrib.gis.gdal.prototypes import ds as vcapi, raster as rcapi
File "/env/lib/python3.5/site-packages/django/contrib/gis/gdal/prototypes/ds.py", line 9, in <module>
from django.contrib.gis.gdal.libgdal import GDAL_VERSION, lgdal
File "/env/lib/python3.5/site-packages/django/contrib/gis/gdal/libgdal.py", line 49, in <module>
lgdal = CDLL(lib_path)
File "/opt/python3.5/lib/python3.5/ctypes/__init__.py", line 351, in __init__
self._handle = _dlopen(self._name, mode)
OSError: /env/lib/python3.5/site-packages/django/contrib/gis/gdal: cannot read file data: Is a directory
Developed on ubuntu 16.04.
I started with simple "pip install django" inside virtualenv, which created the "django/contrib/gis/gdal" folder that says is missing on deploy. I've tried several others ways of installing GDAL to make sure there's no missing header file. Also upgraded to Google Cloud SDK 171 and gsutil 4.27. Have tried including GDAL==2.1.3 in requirements.txt, but then got a build error, so removed it. I also tried "pip freeze" to make sure requirements.txt had all of the needed packages.
Finally, I started again from scratch following this tutorial: https://cloud.google.com/python/django/flexible-environment. I didn't test on a local postgreSQL db, but went directly to the Cloud SQL instance previously used (which had the tables needed). And this time I used python2.7.
It still runs fine locally, but is now missing the folder "/env/local/lib/python2.7/site-packages/django/contrib/gis/gdal" in the stack trace. Here's the full trace:
Traceback (most recent call last):
File "/env/local/lib/python2.7/site-packages/gunicorn/arbiter.py", line 578, in spawn_worker
worker.init_process()
File "/env/local/lib/python2.7/site-packages/gunicorn/workers/base.py", line 126, in init_process
self.load_wsgi()
File "/env/local/lib/python2.7/site-packages/gunicorn/workers/base.py", line 135, in load_wsgi
self.wsgi = self.app.wsgi()
File "/env/local/lib/python2.7/site-packages/gunicorn/app/base.py", line 67, in wsgi
self.callable = self.load()
File "/env/local/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 65, in load
return self.load_wsgiapp()
File "/env/local/lib/python2.7/site-packages/gunicorn/app/wsgiapp.py", line 52, in load_wsgiapp
return util.import_app(self.app_uri)
File "/env/local/lib/python2.7/site-packages/gunicorn/util.py", line 352, in import_app
__import__(module)
File "/home/vmagent/app/mysite/wsgi.py", line 22, in <module>
application = get_wsgi_application()
File "/env/local/lib/python2.7/site-packages/django/core/wsgi.py", line 13, in get_wsgi_application
django.setup(set_prefix=False)
File "/env/local/lib/python2.7/site-packages/django/__init__.py", line 27, in setup
apps.populate(settings.INSTALLED_APPS)
File "/env/local/lib/python2.7/site-packages/django/apps/registry.py", line 108, in populate
app_config.import_models()
File "/env/local/lib/python2.7/site-packages/django/apps/config.py", line 202, in import_models
self.models_module = import_module(models_module_name)
File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
File "/env/local/lib/python2.7/site-packages/django/contrib/auth/models.py", line 4, in <module>
from django.contrib.auth.base_user import AbstractBaseUser, BaseUserManager
File "/env/local/lib/python2.7/site-packages/django/contrib/auth/base_user.py", line 52, in <module>
class AbstractBaseUser(models.Model):
File "/env/local/lib/python2.7/site-packages/django/db/models/base.py", line 124, in __new__
new_class.add_to_class('_meta', Options(meta, app_label))
File "/env/local/lib/python2.7/site-packages/django/db/models/base.py", line 331, in add_to_class
value.contribute_to_class(cls, name)
File "/env/local/lib/python2.7/site-packages/django/db/models/options.py", line 214, in contribute_to_class
self.db_table = truncate_name(self.db_table, connection.ops.max_name_length())
File "/env/local/lib/python2.7/site-packages/django/db/__init__.py", line 33, in __getattr__
return getattr(connections[DEFAULT_DB_ALIAS], item)
File "/env/local/lib/python2.7/site-packages/django/db/utils.py", line 211, in __getitem__
backend = load_backend(db['ENGINE'])
File "/env/local/lib/python2.7/site-packages/django/db/utils.py", line 115, in load_backend
return import_module('%s.base' % backend_name)
File "/usr/lib/python2.7/importlib/__init__.py", line 37, in import_module
__import__(name)
File "/env/local/lib/python2.7/site-packages/django/contrib/gis/db/backends/postgis/base.py", line 6, in <module>
from .features import DatabaseFeatures
File "/env/local/lib/python2.7/site-packages/django/contrib/gis/db/backends/postgis/features.py", line 1, in <module>
from django.contrib.gis.db.backends.base.features import BaseSpatialFeatures
File "/env/local/lib/python2.7/site-packages/django/contrib/gis/db/backends/base/features.py", line 4, in <module>
from django.contrib.gis.db.models import aggregates
File "/env/local/lib/python2.7/site-packages/django/contrib/gis/db/models/__init__.py", line 3, in <module>
from django.contrib.gis.db.models.aggregates import * # NOQA
File "/env/local/lib/python2.7/site-packages/django/contrib/gis/db/models/aggregates.py", line 1, in <module>
from django.contrib.gis.db.models.fields import ExtentField
File "/env/local/lib/python2.7/site-packages/django/contrib/gis/db/models/fields.py", line 3, in <module>
from django.contrib.gis import forms, gdal
File "/env/local/lib/python2.7/site-packages/django/contrib/gis/forms/__init__.py", line 3, in <module>
from .fields import ( # NOQA
File "/env/local/lib/python2.7/site-packages/django/contrib/gis/forms/fields.py", line 4, in <module>
from django.contrib.gis.geos import GEOSException, GEOSGeometry
File "/env/local/lib/python2.7/site-packages/django/contrib/gis/geos/__init__.py", line 5, in <module>
from .collections import ( # NOQA
File "/env/local/lib/python2.7/site-packages/django/contrib/gis/geos/collections.py", line 11, in <module>
from django.contrib.gis.geos.geometry import GEOSGeometry, LinearGeometryMixin
File "/env/local/lib/python2.7/site-packages/django/contrib/gis/geos/geometry.py", line 11, in <module>
from django.contrib.gis import gdal
File "/env/local/lib/python2.7/site-packages/django/contrib/gis/gdal/__init__.py", line 28, in <module>
from django.contrib.gis.gdal.datasource import DataSource
File "/env/local/lib/python2.7/site-packages/django/contrib/gis/gdal/datasource.py", line 39, in <module>
from django.contrib.gis.gdal.driver import Driver
File "/env/local/lib/python2.7/site-packages/django/contrib/gis/gdal/driver.py", line 5, in <module>
from django.contrib.gis.gdal.prototypes import ds as vcapi, raster as rcapi
File "/env/local/lib/python2.7/site-packages/django/contrib/gis/gdal/prototypes/ds.py", line 9, in <module>
from django.contrib.gis.gdal.libgdal import GDAL_VERSION, lgdal
File "/env/local/lib/python2.7/site-packages/django/contrib/gis/gdal/libgdal.py", line 45, in <module>
% '", "'.join(lib_names)
ImproperlyConfigured: Could not find the GDAL library (tried "gdal", "GDAL", "gdal2.1.0", "gdal2.0.0", "gdal1.11.0", "gdal1.10.0", "gdal1.9.0"). Is GDAL installed? If it is, try setting GDAL_LIBRARY_PATH in your settings.
Any help is greatly appreciated.
Dan
The app engine flexible environment comes with the C libraries listed here: https://cloud.google.com/appengine/docs/flexible/python/runtime
Since GDAL is a C library that isn't in the runtimes that come standard, I had to build a custom environment using a Docker container. The trick was getting GDAL installed along with the necessary python libraries in requirements.txt and Python3, then starting the django app thru gunicorn.
Here's my Dockerfile:
FROM gcr.io/google-appengine/python
RUN apt-get update && apt-get install -y \
binutils \
gdal-bin \
python-gdal
# Create a virtualenv for dependencies. This isolates these packages from
# system-level packages.
RUN virtualenv /env -p python3.6
# Setting these environment variables are the same as running
# source /env/bin/activate.
ENV VIRTUAL_ENV /env
ENV PATH /env/bin:$PATH
# Copy the application's requirements.txt and run pip to install all
# dependencies into the virtualenv.
ADD requirements.txt /app/requirements.txt
RUN pip install -r /app/requirements.txt
# Add the application source code.
ADD . /app
# Run a WSGI server to serve the application. gunicorn must be declared as
# a dependency in requirements.txt.
CMD gunicorn -b :$PORT mysite.wsgi
My app.yaml:
# [START runtime]
runtime: custom
#python
env: flex
entrypoint: gunicorn -b :$PORT mysite.wsgi
beta_settings:
cloud_sql_instances: <your-db-instance-identifying-string>
runtime_config:
python_version: 3
# [END runtime]
And the minimum requirements.txt (add to as needed):
Django==1.11.4
mysqlclient==1.3.10
wheel==0.29.0
gunicorn==19.7.1
psycopg2==2.7.3
Hope this helps someone,
Dan
来源：https://stackoverflow.com/questions/46348004/gdal-in-flexible-environment