Django prefetch_related with limit

Question

Is there a way to tell prefetch_related to only fetch a limited set of related objects? Lets say I am fetching a list of users and I know I want to fetch their

Answer 1

thats what actually works for me django(2.1) (based on haseebahmad answer).
in order for prefetch_related to accept customize queryset: Prefetch
so:
from django.db.models import OuterRef, Subquery ,Prefetch

User.objects.all().prefetch_related(Prefetch('comment_set',  
queryset=Comment.objects.filter(id__in= 
Subquery(Comment.objects.filter(user_id=OuterRef('user_id')).
values_list('id', flat=True)[:1]))))