Google App Engine HRD - what if I exceed the 1 write per second limit for writing to the entity group?

Question

According to the Google App Engine documentation, writing to one entity group is limited to one write per second when using High Replication Datastore. So...

1. What happens if I exceed this limit? Some kind of exception? And what should I do?
2. How do I know that I'm close to exceeding this limit? I can design the application in a way that particular actions (adding an entity...) are not likely to happen often but naturally I can't guarantee that.

Answer 1

one write per second is a little low. it actually supports more than that. i would even say between 5 and 10 writes per second but i can't guarantee that of course.

if you hit that limit you will get an exception yes. the exception message will be:
Too much contention on these datastore entities. please try again.

but i don't know/remember the exact Exception that will raise.

you have the choice to retry, continue or whatever else you think is right to to at that point.

you can't tell if you are close to a 1 write per second limit. it happens and you deal with it.

Answer 2

Based on the GAE documentation and from my limited experience:

1. Expect something like 1 QPS rate and tune your app accordingly.
2. Sharding is a common pattern to handle datastore contention.
3. Always add defensive code to handle every possible exception (Application Error 5, The datastore operation timed out, Transaction collision for entity group, ..)
4. In case of error, retry the write moving the task in a proper taskqueue or, if you can, just alert the user to try again.
5. Retrying a write, usually works.
6. When possible, use a write-behind cache mechanism moving the writes operation that can lead to contention to Memcache and a Taskqueue slowing down the datastore hit rate.
7. A good way to avoid contention is to keep the entity groups small, but don't count on it too much.
8. You can have contention even on single entity.