How to update an NDB Model's Schema [closed]

Question

Closed. This question needs details or clarity. It is not currently accepting answers.

---

Want to improve this question? Add details and clarify the problem by editing this post.

Closed 4 years ago.

I have seen the solution to this question using App Engine's older DB Datastore API, but cannot find a solution while using the newer NDB API.

What is the best way to add migration support, so that I am able to migrate from an old version of a schema to a new version. Would it be best to write a migration script, and how would this work?

Something like migrating a schema like this in (Note that the sample is in NDB):

class Picture(ndb.Model):
    author = ndb.StringProperty()
    png_data = ndb.BlobProperty()
    name = ndb.StringProperty(default='')  # Unique name.

To an updated one like this:

class Picture(ndb.Model):
    author = ndb.StringProperty()
    png_data = ndb.BlobProperty()
    name = ndb.StringProperty(default='')  # Unique name.

    num_votes = ndb.IntegerProperty(default=0)
    avg_rating = ndb.FloatProperty(default=0)

Many thanks!

Answer 1

So far as I know, NDB doesn't have any built-in schema migration support. We handle schema migrations this way:

• Add an NDB integer property called schema_version to each Model we use. This will be the version of schema that was active when the model was stored.
• Add a class attribute SCHEMA_VERSION to each Model, representing the current schema version of the code. It defaults to 0, and we bump this up whenever we add/remove/change NDB properties on the model.
• Have our Models implement a updateSchema method that can look at the stored and current versions and perform the appropriate migrations.
• During load of entities, we check to see if the schema version of the entity we loaded is out of date. If it is, we call updateSchema to fix up the entity before we return it from our data layer. We then set schema_version to the value of SCHEMA_VERSION.

This approach means that we are updating the schema on-demand. If we run into a situation where we really want all entities of a particular type to be updated now, we write a map/reduce operation that loads and saves each entity of that type; the schema migration happens automatically as a part of that process, without incurring downtime.

Now, this works unless you are dealing with models with model superclasses whose schemas can also change. To address that, you need to collect the different values of SCHEMA_VERSION up the class hierarchy when you come up with a value to store in schema_version. We collect them by simply summing them up to come up with the official "current schema version" of the entity, but other ways of doing this are possible.