app-engine-ndb

Google App Engine error: NeedIndexError: no matching index found

那年仲夏 提交于 2019-11-27 15:52:43
问题 I'm having trouble with Google's App engine indexes. When running my app via the GoogleAppEngineLauncher, the app is working fine. When deploying the app, I get the following error: NeedIndexError: no matching index found. The suggested index for this query is: - kind: Bar ancestor: yes properties: - name: rating direction: desc The error is generated after this line of code: bars = bar_query.fetch(10) Before the above line of code, it reads: bar_query = Bar.query(ancestor=guestbook_key

Memory leak in Google ndb library

核能气质少年 提交于 2019-11-27 14:11:30
I think there is a memory leak in the ndb library but I can not find where. Is there a way to avoid the problem described below? Do you have a more accurate idea of testing to figure out where the problem is? That's how I reproduced the problem : I created a minimalist Google App Engine application with 2 files. app.yaml : application: myapplicationid version: demo runtime: python27 api_version: 1 threadsafe: yes handlers: - url: /.* script: main.APP libraries: - name: webapp2 version: latest main.py : # -*- coding: utf-8 -*- """Memory leak demo.""" from google.appengine.ext import ndb import

how to cleanly remove ndb properties

送分小仙女□ 提交于 2019-11-27 10:59:05
问题 in my app i need to remove a few of my models properties. i checked out this link but the first issue is that the properties are on a polymodel and there is no way im going to switch to an expando for the time to remove the properties, im not even shure what could happen if i change a polymodel to an expando . so how do i remove properties from existing entities? i was thinking to set all StringProperty to None and then remove these from the model schema and redeploy. one of those properties

Google appengine: Task queue performance

断了今生、忘了曾经 提交于 2019-11-27 07:26:55
问题 I currently have an application running on appengine and I am executing a few jobs using the deferred library, some of these tasks run daily, while some of them are executed once a month. Most of these tasks query Datastore to retrieve documents and then store the entities in an index (Search API). Some of these tables are replaced monthly and I have to run these tasks on all entities (4~5M). One exemple of such a task is: def addCompaniesToIndex(cursor=None, n_entities=0, mindate=None): #get

GAE ndb design, performance and use of repeated properties

情到浓时终转凉″ 提交于 2019-11-27 01:59:19
Say I have a picture gallery and a picture could potentially have 100k+ fans. Which ndb design is more efficient? class picture(ndb.model): fanIds = ndb.StringProperty(repeated=True) ... [other picture properties] or class picture(ndb.model): ... [other picture properties] class fan(ndb.model): pictureId = StringProperty() fanId = StringProperty() Is there any limit on the number of items you can add to an ndb repeated property and is there any performance hit with storing a large amount of items in a repeated property? If it is less efficient to use repeated properties, what is their intended

Migrating data when changing an NDB field's property type

天大地大妈咪最大 提交于 2019-11-27 01:46:57
Suppose I initially create an ndb.Model and wanted to change a field's ndb property type (e.g. IntegerProperty to StringProperty), but wanted to cast the current data stored in that field so that I don't lose that data. One method would be to simply create a new field name and then migrate the data over with a script, but are there other more convenient ways of accomplishing this? For example, suppose I had the following model: class Car(ndb.Model): name = ndb.StringProperty() production_year = ndb.IntegerProperty() And I stored an instance of the entity: c = new Car() c.name = "Porsche" c

Best practice to query large number of ndb entities from datastore

↘锁芯ラ 提交于 2019-11-26 23:46:40
问题 I have run into an interesting limit with the App Engine datastore. I am creating a handler to help us analyze some usage data on one of our production servers. To perform the analysis I need to query and summarize 10,000+ entities pulled from the datastore. The calculation isn't hard, it is just a histogram of items that pass a specific filter of the usage samples. The problem I hit is that I can't get the data back from the datastore fast enough to do any processing before hitting the query

Contention problems in Google App Engine

家住魔仙堡 提交于 2019-11-26 09:58:29
问题 I\'m having contention problems in Google App Engine, and try to understand what\'s going on. I have a request handler annotated with: @ndb.transactional(xg=True, retries=5) ..and in that code I fetch some stuff, update some others etc. But sometimes an error like this one comes in the log during a request: 16:06:20.930 suspended generator _get_tasklet(context.py:329) raised TransactionFailedError(too much contention on these datastore entities. please try again. entity group key: app: \"s~my

GAE ndb design, performance and use of repeated properties

蹲街弑〆低调 提交于 2019-11-26 09:51:28
问题 Say I have a picture gallery and a picture could potentially have 100k+ fans. Which ndb design is more efficient? class picture(ndb.model): fanIds = ndb.StringProperty(repeated=True) ... [other picture properties] or class picture(ndb.model): ... [other picture properties] class fan(ndb.model): pictureId = StringProperty() fanId = StringProperty() Is there any limit on the number of items you can add to an ndb repeated property and is there any performance hit with storing a large amount of

Migrating data when changing an NDB field's property type

拟墨画扇 提交于 2019-11-26 09:47:20
问题 Suppose I initially create an ndb.Model and wanted to change a field\'s ndb property type (e.g. IntegerProperty to StringProperty), but wanted to cast the current data stored in that field so that I don\'t lose that data. One method would be to simply create a new field name and then migrate the data over with a script, but are there other more convenient ways of accomplishing this? For example, suppose I had the following model: class Car(ndb.Model): name = ndb.StringProperty() production