app-engine-ndb

TransactionFailedError (too much contention…) when reading (cross-group) entities from datastore

此生再无相见时 提交于 2019-12-02 04:34:25
I’m investigating again the unexpected occurrence of TransactionFailedError (too much contention on these datastore entities... in cases, where the code only reads entity groups that are blamed for the contention problems. Setup GAE standard environment, Python 2.7 with NDB (SDK 1.9.51). I managed to observe the error in an isolated app (only me as user) where the same request handler is executed in a task queue and read/write access to the entity-groups mentioned below is only done by this handler. The handler is executed a few times per second and basically is a migration / copy task to move

Using Key in NDB to retrieve an entity

牧云@^-^@ 提交于 2019-12-02 03:48:12
I have this structure: Books that have Chapters (ancestor=Book) that have Pages (ancestor=Chapter) It is clear for me that, to search for a Chapter by ID, I need the book to search by ancestor query. And I've learn today that if I have all the keys, I can retrieve directly the entity without need of get first the book, then the chapter and then the page, in this way: page_key = ndb.Key('Book', long(bookId), 'Chapter', long(chapterId), 'Page', long(pageId)) page = page_key.get() My doubt is: to get the page by, for example, page number, must I get first the chapter? For example : class Book(ndb

NDB query fetch() and ContextOptions

我与影子孤独终老i 提交于 2019-12-02 02:40:16
问题 I would like to disable context cache in only one of my queries. I thought I could do it like this: MyModel.query(ancestor=user.key).fetch(100, options=ContextOptions(use_cache=False, use_memcache=False)) or MyModel.query(ancestor=user.key).fetch(100, config=ContextOptions(use_cache=False, use_memcache=False)) But it does not seem to work for me. So my question is how do I disable cache and memcache for queries using fetch ? PS: For get() it works perfectly: MyModel.query(ancestor=user.key)

In Google App Engine, how to check input validity of Key created by urlsafe?

爱⌒轻易说出口 提交于 2019-12-02 01:08:16
问题 Suppose I create a key from user input websafe url key = ndb.Key(urlsafe=some_user_input) How can I check if the some_user_input is valid? My current experiment shows that statement above will throw ProtocolBufferDecodeError (Unable to merge from string.) exception if the some_user_input is invalid, but could not find anything about this from the API. Could someone kindly confirm this, and point me some better way for user input validity checking instead of catching the exception? Thanks a

In Google App Engine, how to check input validity of Key created by urlsafe?

老子叫甜甜 提交于 2019-12-01 21:03:21
Suppose I create a key from user input websafe url key = ndb.Key(urlsafe=some_user_input) How can I check if the some_user_input is valid? My current experiment shows that statement above will throw ProtocolBufferDecodeError (Unable to merge from string.) exception if the some_user_input is invalid, but could not find anything about this from the API. Could someone kindly confirm this, and point me some better way for user input validity checking instead of catching the exception? Thanks a lot! If you try to construct a Key with an invalid urlsafe parameter key = ndb.Key(urlsafe='bogus123')

NDB Caching When Using Projected Queries

99封情书 提交于 2019-12-01 13:27:46
Could not find this specific question asked before yet. How does App Engine's NDB handle caching when using projected queries vs. full entity queries? For example, if I do a projected query first.. MyModel.query().get(projected=['name']) ...and then do a regular query next... MyModel.query().get() ...what will I get? The full entity? If so, was ANY part of first query automatically cached by NDB? Or is NDB able to make the distinction well, so the next time I run the projected query it is potentially pulled from cache? As far as I'm aware, query results are only cached in in-context cache, but

NDB Caching When Using Projected Queries

最后都变了- 提交于 2019-12-01 10:35:24
问题 Could not find this specific question asked before yet. How does App Engine's NDB handle caching when using projected queries vs. full entity queries? For example, if I do a projected query first.. MyModel.query().get(projected=['name']) ...and then do a regular query next... MyModel.query().get() ...what will I get? The full entity? If so, was ANY part of first query automatically cached by NDB? Or is NDB able to make the distinction well, so the next time I run the projected query it is

How to programmatically determine if ndb property is multivalue

会有一股神秘感。 提交于 2019-12-01 09:16:31
I am translating an app from Datastore to ndb and have encountered a problem in the xml import routine. The problem is that I am not able to programmatically determine whether a property of a ndb.model class is a multivalue property or not. I suspect that this is due to lack of basic Python skills since the code I have come up with so far shows that the value is “visible”. I am thus not able to grab it. Please help. from google.appengine.ext import ndb class House(ndb.Model): name = ndb.StringProperty() rooms = ndb.StringProperty(repeated=True) print 'Properties:' for p in House._properties:

How do I access my AppEngine DataStore entities from my Compute Engine VM?

蹲街弑〆低调 提交于 2019-12-01 09:14:32
My app is running on App Engine, but I would like to access its NDB DataStore entities from my Compute Engine VM to do some processing and write the results back to the App Engine DataStore. How can I do that? Also, are the Google Cloud DataStore and App Engine DataStore the same thing? https://developers.google.com/datastore/ https://developers.google.com/appengine/docs/python/ndb/ Dmytro Sadovnychyi David's solution requires you to use App Engine instance time to make requests, but you can bypass it and make requests directly to Datastore from Compute Engine instance. There is a pretty good

TransactionFailedError (too much contention…) when reading (cross-group) entities from datastore

浪尽此生 提交于 2019-12-01 08:53:29
问题 I’m investigating again the unexpected occurrence of TransactionFailedError (too much contention on these datastore entities... in cases, where the code only reads entity groups that are blamed for the contention problems. Setup GAE standard environment, Python 2.7 with NDB (SDK 1.9.51). I managed to observe the error in an isolated app (only me as user) where the same request handler is executed in a task queue and read/write access to the entity-groups mentioned below is only done by this