How to retrieve more than 1000 rows from Parse.com?

Question

I have been using Parse to retrieve a data for a list view. Unfortunately they limit requests to 100 by default to a 1000 max. I have well over that 1000 max in my class. I

Answer 1

Here is a JavaScript version without promises..

These are the global variables (collections are not required, just a bad habit of mine)..

   ///create a collection of cool things and instantiate it (globally)
    var CoolCollection = Parse.Collection.extend({
       model: CoolThing
    }), coolCollection = new CoolCollection();

This is the "looping" function that gets your results..

//recursive call, initial loopCount is 0 (we haven't looped yet)
function getAllRecords(loopCount){

    ///set your record limit
    var limit = 1000;

    ///create your eggstra-special query
     new Parse.Query(CoolThings)
            .limit(limit)
            .skip(limit * loopCount) //<-important
            .find({
             success: function (results) {
                 if(results.length > 0){

                     //we do stuff in here like "add items to a collection of cool things"
                     for(var j=0; j < results.length; j++){
                         coolCollection.add(results[j]);
                     }

                     loopCount++; //<--increment our loop because we are not done

                     getAllRecords(loopCount); //<--recurse
                 }
                 else
                 {
                     //our query has run out of steam, this else{} will be called one time only
                     coolCollection.each(function(coolThing){
                        //do something awesome with each of your cool things
                     });
                 }
            },
             error: function (error) {
                //badness with the find
             }
         });
}

This is how you call it (or you could do it other ways):

getAllRecords(0);

Answer 2

GENERIC VERSION For SWIFT 4:

Warning: this is not tested!

An attempt to adapt nyxee's answer to be usable for any query:

func getAllRecords(for query: PFQuery<PFObject>, then doThis: @escaping (_ objects: [PFObject]?, _ error: Error?)->Void) {
    let limit = 1000
    var objectArray : [PFObject] = []
    query.limit = limit
    func recursiveQuery(_ loopCount:  Int = 0){
        query.skip  = limit * loopCount
        query.findObjectsInBackground(block: { (objects, error) in
            if let objects = objects {
                objectArray.append(contentsOf: objects)
                if objects.count == limit {
                    recursiveQuery(loopCount + 1)
                } else {
                    doThis(objectArray, error)
                }
            } else {
                doThis(objects, error)
            }
        })
    }
    recursiveQuery()
}

Answer 3

A Swift 3 Example:

var users    = [String] ()
var payments = [String] ()
///set your record limit
let limit = 29
//recursive call, initial loopCount is 0 (we haven't looped yet)
func loadAllPaymentDetails(_ loopCount:  Int){
    ///create your NEW eggstra-special query
    let paymentsQuery  = Payments.query()
    paymentsQuery?.limit = limit
    paymentsQuery?.skip  = limit*loopCount
    paymentsQuery?.findObjectsInBackground(block: { (objects, error) in
        if let objects = objects {
            //print(#file.getClass(),"  ",#function,"  loopcount: ",loopCount,"      #ReturnedObjects: ",  objects.count)
            if objects.count > 0 {
                //print(#function, " no. of objects :", objects.count)
                for paymentsObject in objects {
                    let user   = paymentsObject[Utils.name] as! String
                    let amount = paymentsObject[Utils.amount] as! String
                    self.users.append(user)
                    self.payments.append(amount)
                }
                //recurse our loop with increment because we are not done
                self.loadAllPaymentDetails(loopCount + 1); //<--recurse
            }else {
                //our query has run out of steam, this else{} will be called one time only
                //if the Table had been initially empty, lets inform the user:
                if self.users.count == 1 {
                    Utils.createAlert(self, title: "No Payment has been made yet", message: "Please Encourage Users to make some Payments", buttonTitle: "Ok")
                }else {
                    self.tableView.reloadData()
                }
            }
        }else if error != nil {
            print(error!)
        }else {
            print("Unknown Error")
        }
    })
}

adapted from @deLux_247's example above.

Answer 4

JAVA

So after 5 years, 4 months the above answer of @SquiresSquire needed some changes to make it work for me, and I would like to share it with you

private static List<ParseObject>allObjects = new ArrayList<ParseObject>();

.

 ParseQuery<ParseObject> parseQuery = new ParseQuery<ParseObject>("CLASSNAME");
        parseQuery.setLimit(1000);
        parseQuery.findInBackground(getAllObjects());

.

FindCallback <ParseObject> getAllObjects() {
        return new FindCallback <ParseObject>() {
            @Override
            public void done(List<ParseObject> objects, ParseException e) {
                if (e == null) {
                   allObjects.addAll(objects);
                    int limit = 1000;
                    if (objects.size() == limit) {
                        skip = skip + limit;
                        ParseQuery query = new ParseQuery("CLASSNAME");
                        query.setSkip(skip);
                        query.setLimit(limit);
                        query.findInBackground(getAllObjects());
                    }
                    //We have a full PokeDex
                    else {
                        //USE FULL DATA AS INTENDED
                    }
                }
            }

        };

Answer 5

In C# I use this recursion:

private static async Task GetAll(int count = 0, int limit = 1000)
{
    if (count * limit != list.Count) return;
    var res = await ParseObject.GetQuery("Row").Limit(limit).Skip(list.Count).FindAsync();
    res.ToList().ForEach(x => list.Add(x));
    await GetAll(++count);
}

JS version:

function getAll(list) {
    new Parse.Query(Row).limit(1000).skip(list.length).find().then(function (result) {
        list = list.concat(result);
        if (result.length != 1000) {
            //do here something with the list...
            return;
        }

        getAll(list);
    });
}

Usage: GetAll() in C#, and getAll([]) in JS.

I store all rows from the class Rowin the list. In each request I get 1000 rows and skip the current size of the list. Recursion stops when the current number of exported rows is different from the expected.

Answer 6

You could achieve this using CloudCode... Make a custom function you can call that will enumerate the entire collection and build a response from that but a wiser choice would be to paginate your requests, and fetch the records 1000 (or even less) at a time, adding them into your list dynamically as required.