How to write a Django QuerySet the properly computes average of DateTimeField with grouping?

Question

Here is my Django model:

class MyModel(models.Model):
    a = IntegerField()
    b = DateTimeField()

Here is the QuerySet I execute on this model to find the count, min, max and average bs for each value of a:

>>> from django.db.models import Count, Max, Min, Avg
>>> MyModel.objects.extra(
...    select={'avg': 'AVG(UNIX_TIMESTAMP(b))'}
... ).values('a').annotate(
...     count=Count('b'), 
...     min=Min('b'), 
...     max=Max('b'),
... )

Here is the result of the QuerySet above:

[
  {'a': 1, 'count': 5, 'min': datetime.datetime(2015, 2, 26, 1, 8, 21, tzinfo=<UTC>), 'max': datetime.datetime(2015, 2, 26, 1, 8, 22, tzinfo=<UTC>)}, 
  {'a': 2, 'count': 2, 'min': datetime.datetime(2015, 2, 26, 1, 8, 21, tzinfo=<UTC>), 'max': datetime.datetime(2015, 2, 26, 1, 8, 22, tzinfo=<UTC>)}
]

As you can see, the results of the QuerySet do not include the average field that I calculated. How can I get that in there? I have tried many different permutations. But if I can get the avg field in there, then it seems to screw up the grouping by a.

Answer 1

Instead of using the Django ORM, you could use a raw sql query.

from django.db import connection
query = """
SELECT `a`,
       COUNT(b) AS `count`,
       MAX(b) AS `max`,
       AVG(UNIX_TIMESTAMP(b)) AS `avg`,
       MIN(b) AS `min`
FROM `<appname>_<modelname>`
GROUP BY `a`
"""
cursor = connection.cursor()
cursor.execute(query)
result = cursor.fetchall()

This will give you something like:

(
    (a value1, count, max, avg as unix timestamp, min), ...
    (a value2, count, max, avg as unix timestamp, min), ...
    (a value3, count, max, avg as unix timestamp, min), ...
)

Otherwise, the closest thing I could get using Django's ORM would be to abandon the UNIX_TIMESTAMP conversion in the extra clause:

from django.db.models import Count, Min, Max, Avg
MyModel.objects.all().values('a').annotate(
    count=Count('b'), 
    max=Max('b'), 
    avg=Avg('b'), 
    min=Min('b')
)

Unfortunately, this will give you the average as a float.

[
    {
        'count': 15366, 
         'a': 0, 
         'avg': 19898862327498.82, 
         'min': datetime.datetime(1900, 1, 1, 0, 0), 
         'max': datetime.datetime(2012, 7, 3, 0, 0)
    }, {
         'count': 1726, 
         'a': 1, 
         'avg': 19785827400927.0, 
         'min': datetime.datetime(1920, 8, 25, 0, 0), 
         'max': datetime.datetime(1994, 12, 29, 0, 0)
    }, 
    ...

]

You can try using something like

import datetime
datetime.datetime.strptime(str(int(ts)), '%Y%m%d%H%M%S%f')

to convert it back to a datetime object, though this will be an approximation, so I recommend using raw sql instead.