问题
I am trying to create an application in appengine that searches for a list of keys and then I use this list to delete these records from the datastore, this service has to be a generic service so I could not use a model just search by the name of kind, it is possible to do this through appengine features?
Below my code, but it requires that I have a model.
import httplib import logging from datetime import datetime, timedelta
import webapp2
from google.appengine.api import urlfetch
from google.appengine.ext import ndb
DEFAULT_PAGE_SIZE = 100000
DATE_PATTERN = "%Y-%m-%dT%H:%M:%S"
def get_date(amount):
date = datetime.today() - timedelta(days=30 * amount)
date = date.replace(hour=0, minute=0, second=0)
return date
class Purge(webapp2.RequestHandler):
def get(self):
kind = self.request.get('kind')
datefield = self.request.get('datefield')
amount = self.request.get('amount', default_value=3)
date = get_date(amount)
logging.info('Executando purge para Entity {}, mantendo periodo de {} meses.'.format(kind, amount))
# cria a query
query = ndb.Query(kind=kind, namespace='development')
logging.info('Setando o filtro [{} <= {}]'.format(datefield, date.strftime(DATE_PATTERN)))
# cria um filtro
query.filter(ndb.DateTimeProperty(datefield) <= date)
query.fetch_page(DEFAULT_PAGE_SIZE)
while True:
# executa a consulta
keys = query.fetch(keys_only=True)
logging.info('Encontrados {} {} a serem exluidos'.format(len(keys), kind))
# exclui usando as keys
ndb.delete_multi(keys)
if len(keys) < DEFAULT_PAGE_SIZE:
logging.info('Nao existem mais registros a serem excluidos')
break
app = webapp2.WSGIApplication(
[
('/cloud-datastore-purge', Purge),
], debug=True)
Trace
Traceback (most recent call last):
File "/base/alloc/tmpfs/dynamic_runtimes/python27g/7894e0c59273b2b7/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1535, in __call__
rv = self.handle_exception(request, response, e)
File "/base/alloc/tmpfs/dynamic_runtimes/python27g/7894e0c59273b2b7/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1529, in __call__
rv = self.router.dispatch(request, response)
File "/base/alloc/tmpfs/dynamic_runtimes/python27g/7894e0c59273b2b7/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1278, in default_dispatcher
return route.handler_adapter(request, response)
File "/base/alloc/tmpfs/dynamic_runtimes/python27g/7894e0c59273b2b7/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 1102, in __call__
return handler.dispatch()
File "/base/alloc/tmpfs/dynamic_runtimes/python27g/7894e0c59273b2b7/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 572, in dispatch
return self.handle_exception(e, self.app.debug)
File "/base/alloc/tmpfs/dynamic_runtimes/python27g/7894e0c59273b2b7/python27/python27_lib/versions/third_party/webapp2-2.5.2/webapp2.py", line 570, in dispatch
return method(*args, **kwargs)
File "/base/data/home/apps/p~telefonica-dev-155211/cloud-datastore-purge-python:20180629t150020.410785498982375644/purge.py", line 38, in get
query.fetch_page(_DEFAULT_PAGE_SIZE)
File "/base/alloc/tmpfs/dynamic_runtimes/python27g/7894e0c59273b2b7/python27/python27_lib/versions/1/google/appengine/ext/ndb/utils.py", line 160, in positional_wrapper
return wrapped(*args, **kwds)
File "/base/alloc/tmpfs/dynamic_runtimes/python27g/7894e0c59273b2b7/python27/python27_lib/versions/1/google/appengine/ext/ndb/query.py", line 1362, in fetch_page
return self.fetch_page_async(page_size, **q_options).get_result()
File "/base/alloc/tmpfs/dynamic_runtimes/python27g/7894e0c59273b2b7/python27/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 383, in get_result
self.check_success()
File "/base/alloc/tmpfs/dynamic_runtimes/python27g/7894e0c59273b2b7/python27/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 427, in _help_tasklet_along
value = gen.throw(exc.__class__, exc, tb)
File "/base/alloc/tmpfs/dynamic_runtimes/python27g/7894e0c59273b2b7/python27/python27_lib/versions/1/google/appengine/ext/ndb/query.py", line 1380, in _fetch_page_async
while (yield it.has_next_async()):
File "/base/alloc/tmpfs/dynamic_runtimes/python27g/7894e0c59273b2b7/python27/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 427, in _help_tasklet_along
value = gen.throw(exc.__class__, exc, tb)
File "/base/alloc/tmpfs/dynamic_runtimes/python27g/7894e0c59273b2b7/python27/python27_lib/versions/1/google/appengine/ext/ndb/query.py", line 1793, in has_next_async
yield self._fut
File "/base/alloc/tmpfs/dynamic_runtimes/python27g/7894e0c59273b2b7/python27/python27_lib/versions/1/google/appengine/ext/ndb/context.py", line 890, in helper
batch, i, ent = yield inq.getq()
File "/base/alloc/tmpfs/dynamic_runtimes/python27g/7894e0c59273b2b7/python27/python27_lib/versions/1/google/appengine/ext/ndb/query.py", line 969, in run_to_queue
batch = yield rpc
File "/base/alloc/tmpfs/dynamic_runtimes/python27g/7894e0c59273b2b7/python27/python27_lib/versions/1/google/appengine/ext/ndb/tasklets.py", line 513, in _on_rpc_completion
result = rpc.get_result()
File "/base/alloc/tmpfs/dynamic_runtimes/python27g/7894e0c59273b2b7/python27/python27_lib/versions/1/google/appengine/api/apiproxy_stub_map.py", line 613, in get_result
return self.__get_result_hook(self)
File "/base/alloc/tmpfs/dynamic_runtimes/python27g/7894e0c59273b2b7/python27/python27_lib/versions/1/google/appengine/datastore/datastore_query.py", line 2951, in __query_result_hook
self.__results = self._process_results(query_result.result_list())
File "/base/alloc/tmpfs/dynamic_runtimes/python27g/7894e0c59273b2b7/python27/python27_lib/versions/1/google/appengine/datastore/datastore_query.py", line 2984, in _process_results
for result in results]
File "/base/alloc/tmpfs/dynamic_runtimes/python27g/7894e0c59273b2b7/python27/python27_lib/versions/1/google/appengine/datastore/datastore_rpc.py", line 194, in pb_to_query_result
return self.pb_to_entity(pb)
File "/base/alloc/tmpfs/dynamic_runtimes/python27g/7894e0c59273b2b7/python27/python27_lib/versions/1/google/appengine/ext/ndb/model.py", line 690, in pb_to_entity
modelclass = Model._lookup_model(kind, self.default_model)
File "/base/alloc/tmpfs/dynamic_runtimes/python27g/7894e0c59273b2b7/python27/python27_lib/versions/1/google/appengine/ext/ndb/model.py", line 3101, in _lookup_model
kind)
KindError: No model class found for kind 'Test'. Did you forget to import it?
回答1:
The problem was found on the line where fetch_page is set.
Removing this line
query.fetch_page(DEFAULT_PAGE_SIZE)
for this
keys = query.fetch(limit=_DEFAULT_LIMIT, keys_only=True)
回答2:
To run a datastore query without a model class available in the environment, you can use the google.appengine.api.datastore.Query class from the low-level datastore API.
See this question for other ideas.
回答3:
If the goal is simply to wipe entities regardless of their kind then you have some options besides specifying the kinds/models yourself.
you could obtain the list of all kinds from your models file and iterate through them, see How to delete all the entries from google datastore?
using the generic datastore client library (not ndb) you could obtain the list of kinds from the datastore itself, see Is there a way to delete all entities from a Datastore namespace Using Python3 (WITHOUT Dataflow)? But I vaguely remember some issues which might translate to that library not working on the 1st generation standard environment anymore, so I'm not 100% certain of this.
来源：https://stackoverflow.com/questions/51106340/query-in-datastore-without-a-model