Atomic increment of a counter in django

Question

I'm trying to atomically increment a simple counter in Django. My code looks like this:

from models import Counter
from django.db import transaction

@transaction.commit_on_success
def increment_counter(name):
    counter = Counter.objects.get_or_create(name = name)[0]
    counter.count += 1
    counter.save()

If I understand Django correctly, this should wrap the function in a transaction and make the increment atomic. But it doesn't work and there is a race condition in the counter update. How can this code be made thread-safe?

Answer 1

New in Django 1.1

Counter.objects.get_or_create(name = name)
Counter.objects.filter(name = name).update(count = F('count')+1)

or using an F expression:

counter = Counter.objects.get_or_create(name = name)
counter.count = F('count') +1
counter.save( update_fields=["count"] )

Remember to Specify Which fields to update, Or you might encounter race conditions on other possible fields of the model!

A topic on the race condition associated with this approach has been added to the official documentation.

Answer 2

In Django 1.4 there is support for SELECT ... FOR UPDATE clauses, using database locks to make sure no data is accesses concurrently by mistake.

Answer 3

If you don't need to know the value of the counter when you set it, the top answer is definitely your best bet:

counter = Counter.objects.get_or_create(name = name)
counter.count = F('count') + 1
counter.save()

This tells your database to add 1 to the value of count, which it can do perfectly well without blocking other operations. The drawback is that you have no way of knowing what count you just set. If two threads simultaneously hit this function, they would both see the same value, and would both tell the db to add 1. The db would end up adding 2 as expected, but you won't know which one went first.

If you do care about the count right now, you can use the select_for_update option referenced by Emil Stenstrom. Here's what that looks like:

from models import Counter
from django.db import transaction

@transaction.atomic
def increment_counter(name):
    counter = (Counter.objects
               .select_for_update()
               .get_or_create(name=name)[0]
    counter.count += 1
    counter.save()

This reads the current value and locks matching rows until the end of the transaction. Now only one worker can read at a time. See the docs for more on select_for_update.

Answer 4

Keeping it simple and building on @Oduvan's answer:

counter, created = Counter.objects.get_or_create(name = name, 
                                                 defaults={'count':1})
if not created:
    counter.count = F('count') +1
    counter.save()

The advantage here is that if the object was created in the first statement, you don't have to do any further updates.

Answer 5

Django 1.7

from django.db.models import F

counter, created = Counter.objects.get_or_create(name = name)
counter.count = F('count') +1
counter.save()

Answer 6

Or if you just want a counter and not a persistent object you can use itertools counter which is implemented in C. The GIL will provide the safety needed.

--Sai