How do I access my AppEngine DataStore entities from my Compute Engine VM?

Question

My app is running on App Engine, but I would like to access its NDB DataStore entities from my Compute Engine VM to do some processing and write the results back to the App

Answer 1

The officially recommended way is to use the datastore client libraries; see https://cloud.google.com/datastore/docs/reference/libraries You need to create a service account, or use the standard compute engine service account, give permission to either all APIs, or to datastore to that service account, and create the compute engine instance to be part of that service account. See here for more information. Then you can do something like:

from google.auth import compute_engine                                                                                      
from google.cloud import datastore                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                                

datastore_client = datastore.Client(project='yourproject',     
                                    credentials=compute_engine.Credentials())                                                                                                                                                                  
q = datastore_client.query(kind='YourEntity')                                                                                    
q.add_filter('field_name', '=', 'HelloThere') 
print list(q.fetch(1))