AppEngine: No module named pyasn1.compat.binary

Question

I keep getting the following error when hitting my AppEngine server:

ERROR    2017-09-20 07:16:06,978 wsgi.py:263] 
Traceback (most recent call last):
  File "/usr/lib/google-cloud-sdk/platform/google_appengine/google/appengine/runtime/wsgi.py", line 240, in Handle
    handler = _config_handle.add_wsgi_middleware(self._LoadHandler())
  File "/usr/lib/google-cloud-sdk/platform/google_appengine/google/appengine/runtime/wsgi.py", line 299, in _LoadHandler
    handler, path, err = LoadObject(self._handler)
  File "/usr/lib/google-cloud-sdk/platform/google_appengine/google/appengine/runtime/wsgi.py", line 96, in LoadObject
    __import__(cumulative_path)
  File "/home/dclochri/Projects/react-seed/server/main.py", line 3, in <module>
    from controllers import organization_controller
  File "/home/dclochri/Projects/react-seed/server/controllers/organization_controller.py", line 4, in <module>
    import rest_controller as rest
  File "/home/dclochri/Projects/react-seed/server/controllers/rest_controller.py", line 4, in <module>
    import google.oauth2.id_token
  File "/home/dclochri/Projects/react-seed/lib/google/oauth2/id_token.py", line 22, in <module>
    from google.auth import jwt
  File "/home/dclochri/Projects/react-seed/lib/google/auth/jwt.py", line 53, in <module>
    from google.auth import _service_account_info
  File "/home/dclochri/Projects/react-seed/lib/google/auth/_service_account_info.py", line 22, in <module>
    from google.auth import crypt
  File "/home/dclochri/Projects/react-seed/lib/google/auth/crypt/__init__.py", line 39, in <module>
    from google.auth.crypt import rsa
  File "/home/dclochri/Projects/react-seed/lib/google/auth/crypt/rsa.py", line 17, in <module>
    from google.auth.crypt import _python_rsa
  File "/home/dclochri/Projects/react-seed/lib/google/auth/crypt/_python_rsa.py", line 27, in <module>
    from pyasn1.codec.der import decoder
  File "/home/dclochri/Projects/react-seed/lib/pyasn1/codec/der/decoder.py", line 7, in <module>
    from pyasn1.type import univ
  File "/home/dclochri/Projects/react-seed/lib/pyasn1/type/univ.py", line 11, in <module>
    from pyasn1.compat import octets, integer, binary
  File "/usr/lib/google-cloud-sdk/platform/google_appengine/google/appengine/tools/devappserver2/python/runtime/sandbox.py", line 1132, in load_module
    raise ImportError('No module named %s' % fullname)
ImportError: No module named pyasn1.compat.binary
INFO     2017-09-20 07:16:06,982 module.py:821] default: "OPTIONS /rest/organizations/create HTTP/1.1" 500 -

My lib dir contains pyasn1 (v0.3.5), and pysasn1 has a compat module with binary.py. It looks like pysasn1 is installed as a dependency by the google-auth module (see requirements.txt below). Dependencies are installed with pip install -r requirements.txt -t lib, as suggested by the GAE sample.

My code so far hardly deviates from the sample docs for using Firebase with GAE here.

Here are some snippets of what I am using:

Python: 2.7.12

gcloud versions:

Google Cloud SDK 171.0.0
alpha 2017.09.11
app-engine-python 1.9.60
beta 2017.09.11
bq 2.0.25
core 2017.09.11
gsutil 4.27

requirements.txt

google-auth==1.1.0
requests==2.18.4
requests-toolbelt==0.7.1

rest_controller.py (snippet)

import json
import webapp2
import google.auth.transport.requests
import google.oauth2.id_token
import requests_toolbelt.adapters.appengine

class RestHandler(webapp2.RequestHandler):
    def dispatch(self):
        """Middleware for all the routes."""
        # Verify the user is authenticated for each request.
        if not self.is_authorized() and self.request.method != 'OPTIONS':
            self.response.set_status(401)
            self.response.write('Unauthorized')
        else:
            super(RestHandler, self).dispatch()

    def get_claims(self):
        """
        Verify the Firebase credentials on the server side via the
        bearer's token.
        """
        auth_headers = self.request.headers['Authorization']
        id_token = auth_headers.split(' ').pop()
        return google.oauth2.id_token.verify_firebase_token(
            id_token, HTTP_REQUEST)

    def get_user_id(self):
        claims = self.get_claims()
        return claims['sub']

    def is_authorized(self):
        """
        Determines if the user is authorized.
        NOTE: We don't have a server-side session, so we are operating based on
        the user's client token. We are basically verifying that the client
        Firebase token is valid by checking it via a server-side request.
        """
        if 'Authorization' in self.request.headers:
            if self.get_claims():
                return True

        return False

Directory Structure:

/
  lib/
    (deps from requirements.txt here)
  server/
    controllers/
      (controllers here)
    models/
      (models here)
    main.py
  src/
    (client code here - js, css, etc)
  app.yaml
  appengine_config.py

This happens on OSX Yosemite and Ubuntu 16.04. Is there something I'm doing wrong with the imports? Is there an issue with the oauth module?

Answer 1

Lots of thanks to Dan Cornilescu! It turns out that my app.yaml config was the culprit, specifically the skip_files directive.

The bad config looked like:

skip_files:
- ^(.git/.*)
- ^.*bin(/.*)?
- ^.*node_modules(/.*)?
- ^.*public(/.*)?
- ^.*src(/.*)?
- ^env$
- ^(.*/)?.*\.pyc$

The updated config looks like:

# Skip any non-essential files for uploading during deploys.
skip_files:
# Defaults, see: https://cloud.google.com/appengine/docs/standard/python/config/appref#skip_files/
- ^(.*/)?#.*#$
- ^(.*/)?.*~$
- ^(.*/)?.*\.py[co]$
- ^(.*/)?.*/RCS/.*$
- ^(.*/)?\..*$
# Custom
- ^(.*/)?.*/bin/.*$
- ^(.*/)?.*/env/.*$
- ^(.*/)?.*/none_modules/.*$
- ^(.*/)?.*/public/.*$
- ^(.*/)?.*/src/.*$

So a combo of bad regex and skipping the wrong files broke the local app server, and fixing it got it up and running again.