Is there a way to delete all entities from a Datastore namespace Using Python3 (WITHOUT Dataflow)?

问题

Need to wipe down a Datastore namespace before test data is uploaded during testing. Using Cloud Datastore API with Python3.

I'm using Datastore with App Engine in Python3. For testing purposes, I have written a script using the Cloud Datastore API to upload several entities of different kinds to datastore. As this is a small project, at the moment there are only 4 kinds and only 2-3 entities per kind.

I want to add to my pipeline a script to wipe down a particular namespace in Datastore that will contain my test data. I want this to run before the upload of the data and testing so the tests can start from a clean slate every time. I'm using cloud builder to upload the entities to datastore and run my tests in a docker container before deploying to app engine.

At the moment the only solutions I can find are to use Dataflow (totally overkill for this I believe), or to remove each entity individually using it's key. I'd prefer to just wipe down the entire namespace if possible.

If anyone has any advice or suggestions on how to do this please let me know!

回答1:

You can write a script in python to delete all the kinds in a particular namespace. Assuming that you know the name of kinds beforehand.

from google.appengine.ext import ndb
from google.appengine.api import namespace_manager

namespace = "PROVIDE_NAMESPACE_HERE"
kind_list = [kind_1,kind_2,kind_3,kind_4]

namespace_manager.set_namespace(namespace) # will set to the namespace provided

for a_kind in kind_list:
    # will fetch keys of all objects in that kind
    kind_keys = a_kind.gql("").fetch(keys_only = True) 
    # will delete all the keys at once
    ndb.delete_multi(kind_keys) 

After deleting all the kinds from a particular namespace your namespace will be visible for around 24 hours in Cloud Datastore and if after that it doesn't contain any kind it will be automatically deleted.

Hope this answers your question!!

回答2:

You should be able to use the same Cloud Datastore API as your app, which can be used even from outside the Google cloud (see How do I use Google datastore for my web app which is NOT hosted in google app engine?). This answer is based on docs only, I didn't actually try it.

You'd need to do something along these lines:

1. find all kinds in your app's datastore (maybe just in the namespace of interest?). This is possible using the datastore metadata, in particular Kind queries:

   query = client.query(kind='__kind__')
   query.keys_only()
   
   kinds = [entity.key.id_or_name for entity in query.fetch()]
   

2. find the keys for the entities of each kind, use regular keys-only queries as you now know the kind. Limit these queries to the namespace of interest. Delete the entities by keys, in batch mode to be more efficient:

   for kind in kinds:
       query = client.query(kind=kind, namespace=NAMESPACE)
       query.keys_only()
       keys = query.fetch()
       client.delete_multi(keys)
   

来源：https://stackoverflow.com/questions/54764032/is-there-a-way-to-delete-all-entities-from-a-datastore-namespace-using-python3