In Google App Engine, how can I work with eventual consistency in handling form submissions?

Question

I\'ve noticed that, with eventual consistency, the common form-processing workflow that I am used to (submit -> create/update record -> redirect -> reload) doesn\'t work. Up

Answer 1

You probably still can use strong consistency, as long as you design your entity groups properly. Reading closely on the doc linked, the restriction is actually only one update per second per entity group.

This is because Google's underlying masterless consistency protocol ("Paxos") guarantees strong consistency only when working within an entity group (being defined as an entire group of keys, under a root key and including that root key, where the eventual parent of all non-root keys is that root key - see Transactions and entity groups for more).

Quoth the manual:

As mentioned above, an entity group is a set of entities connected through ancestry to a common root element. The organization of data into entity groups can limit what transactions can be performed:

... [ details about transactions ] ...

• There is a write throughput limit of about one transaction per second within a single entity group. This limitation exists because Datastore performs masterless, synchronous replication of each entity group over a wide geographic area to provide high reliability and fault tolerance.

So, if you architect your Datastore keys to use, say, the logged-in-user's ID or email address as a parent key, reading the resulting key on the other end can be guaranteed strongly consistent by specifying that key as an ancestor.

It's also worth noting that an entity doesn't technically have to exist at that parent key - you just need a key to start the root of your entity group so that you can apply your consistency guarantee.

If you can architect that way, you can both guarantee consistency and apply writes at one-per-second-per-user, or whatever you scope your keys to.

For more info, see "Structuring for Strong Consistency".