NDB querying a GenericProperty in repeated Expando StructuredProperty

Question

Hey guys im trying to figure out how to structure my query for the following case

First i have a model defined

class Variant(ndb.Expando):
    test = ndb.StringProperty()


class Item(ndb.Model):
    test2 = ndb.StringProperty()
    variants = ndb.StructuredProperty(Variant, repeated=True)

variant = Variant(test="test", dynamic="a")
item = Item(test2="test", variants=[variant, ])
item.put()

and then for the query stuff.. So far i've tried

dynamic = "dynamic"
Item.query(ndb.GenericProperty("variants.%s" % dynamic) == "a")
Item.query(Item._properties["variants.%s" % dynamic] == "a")
Item.query(getattr(Item.variants, dynamic) == "a")
Item.query(getattr(Item, "variants.%s" % dynamic) == "a")
Item.query(ndb.query.FilterNode("variants.%s" % dynamic, "=", "a"))

generic_prop = ndb.GenericProperty()
generic_prop._name = "variants.%s" % dynamic
Item.query(generic_prop == "a")

and none of these works.. This should be perfectly possible since the property name in the datastore is

variants.dynamic = ["a", ]

Thank you for your help

Answer 1

It's easy using GQL:

Item.gql("WHERE variants.dynamic = 'a'").fetch()

Also this works:

s = StringProperty()
s._name = 'variants.dynamic')
Item.query(s == 'a').fetch()

Please do file a feature request; however it's going to be a balancing act. What syntax would you like to use?

UPDATE:

The same thing works with GenericProperty(), or any other Property subclass.

The reason that GenericProperty('variants.dynamic') is forbidden is to prevent people from doing hacks like this:

class MyHack(ndb.Model):
  foo = StringProperty('bar.baz')

which will confuse the serialization and deserialization code.

Maybe we can add a flag to Property that skips this check but then disallows using the property in a model definition (it would only allow it in a query).

Or maybe we can make this work (I think this would be hard though):

Item.query(Item.variants.dynamic == 'a').fetch()

(only if variants is an Expando).

Answer 2

You can do this with a bit of magic.

SHORT ANSWER:

variants_dynamic_property = ndb.GenericProperty()
variants_dynamic_property._name = 'variants.dynamic'
q = Item.query(variants_dynamic_property == 'a')

LONG ANSWER:

Since you are querying for a GenericProperty, you'll need to create one as the docs state, e.g.:

FlexEmployee.query(ndb.GenericProperty('location') == 'SF')

Similarly, when querying for a StucturedProperty, the docs state you can use a property of the property, e.g.:

Contact.query(Contact.address.city == 'Amsterdam')

So combining these, you would need

Item.query(ndb.GenericProperty('variants.dynamic') == 'a')

but trying to construct the property via ndb.GenericProperty('variants.dynamic') results in the following exception:

  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/ndb/utils.py", line 136, in positional_wrapper
    return wrapped(*args, **kwds)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/ndb/model.py", line 2366, in __init__
    super(GenericProperty, self).__init__(name=name, **kwds)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/ndb/utils.py", line 136, in positional_wrapper
    return wrapped(*args, **kwds)
  File "/Applications/GoogleAppEngineLauncher.app/Contents/Resources/GoogleAppEngine-default.bundle/Contents/Resources/google_appengine/google/appengine/ext/ndb/model.py", line 765, in __init__
    raise ValueError('Name %r cannot contain period characters' % (name,))
ValueError: Name 'variants.dynamic' cannot contain period characters

But you can get around this by using the constructor with no property name and then setting the name after the fact:

variants_dynamic_property = ndb.GenericProperty()
variants_dynamic_property._name = 'variants.dynamic'

Answer 3

Hackish Solution:

A temporary solution: Using a ComputedProperty to query by - in our case:

(adding the following to the model definition)

computed_prop = ndb.ComputedProperty(lambda self: self.repeating_prop[0].sub_prop if self.repeating_prop else None)