Query with paging by cursor causes error because of limitations for “IN filter” in cursor() method… What should be the alternative?

Question

I am developing a twitter like microblogging system by using the following models:

class Member(db.Model):    
    user = db.UserProperty(required=True)
             


        

Answer 1

My solution is using the date value like a cursor as I had described as a comment to Nick Johnson's answer... It is like this:

if cursor: # This is not actually a cursor! It is base64 datetime string
  cursordate = _strptime(base64.b64decode(cursor)) # _strptime is a local method that converts str to datetime

# IN has a limit for lists: 30 items allowed
listofNewsLists = []
listofMemberLists = [followed_member_list[i:i+30] for i in range(0, len(followed_member_list), 30)]
for eachList in listofMemberLists:
   query = NewsItem.all()
   query.filter('posted_by IN', eachList).filter('status =', 1)
   if cursor:
      query.filter('posted_on <', cursordate)
   query.order('-posted_on')                        
   listofNewsLists.append(query.fetch(PAGE_SIZE))

  newsList = []
  if listofNewsLists:
    emptyListCount = 0
    while len(newsList) < PAGE_SIZE and emptyListCount < len(listofNewsLists):
      max = datetime.datetime.min
      maxInd = -1
      emptyListCount = 0
      for i in range(len(listofNewsLists)):
        if listofNewsLists[i] == []:
          emptyListCount += 1
        elif listofNewsLists[i][0].posted_on > max:
          max = listofNewsLists[i][0].posted_on
          maxInd = i
      if max > datetime.datetime.min:
        newsList.append(listofNewsLists[maxInd].pop(0))

template_values['cursor'] = base64.b64encode(newsList[-1].posted_on.isoformat())

That is; I store the last displayed item's date value as the starting point of the new list...

This works well (I guess) unless I have items with same posted_on value...