Taking the example from: http://docs.djangoproject.com/en/dev/topics/db/aggregation/#filter-and-exclude
Publisher.objects.filter(book__rating__gt=3.0).annota
You can use the annotation variable in the filter.
publishers=Publisher.objects.annotate(num_books=Count('book')).filter(num_books__gte=2)