Unable to access BigQuery from local App Engine development server

Question

This is specifically a question relating to server to server authorisation between a python Google AppEngine app and Google\'s BigQuery, but could be relevant for other clou

Answer 1

A recent release of Google App Engine SDK added support for the AppAssertionCredentials method on the development server. To use this method locally, add the following arguments to dev_appserver.py:

$ dev_appserver.py --help
...
Application Identity:
  --appidentity_email_address APPIDENTITY_EMAIL_ADDRESS
                        email address associated with a service account that
                        has a downloadable key. May be None for no local
                        application identity. (default: None)
  --appidentity_private_key_path APPIDENTITY_PRIVATE_KEY_PATH
                        path to private key file associated with service
                        account (.pem format). Must be set if
                        appidentity_email_address is set. (default: None)

To use these:

1. In Google Developer Console, select a project then navigate to "API & auth" -> "Credentials" -> "Create new client ID".

2. Select "Service account" and follow the prompts to download the private key in PKCS12 (.p12) format. Take note of the email address for the service account.

3. Make sure you add that service account email address to the "Permissions" tab for any project that contains data it needs to access, by default it is added to the project team in which it was created.

4. Convert the PKCS12 format to PKCS1 format using the following command:

   $ cat /path/to/xxxx-privatekey.p12 | openssl pkcs12 -nodes -nocerts -passin pass:notasecret | openssl rsa > /path/to/secret.pem

5. Start dev_appserver.py as:

   $ dev_appserver.py --appidentity_email_address xxxx@developer.gserviceaccount.com --appidentity_private_key_path /path/to/secret.pem ...

6. Use appidentity module and AppAssertionCredentials in the same manner locally as you normally would in production.

Please ensure that /path/to/secret.pem is outside of your application source directory so that it is not accidentally deployed as part of your application.