Sort and Group in one MongoDB aggregation query

Question

Using $sort and $group in one aggregation query behaving strangely.

Test data:

db.createCollection("test");

db.test.insert({
    ts : 100,
    category : 1
});

db.test.insert({
    ts : 80,
    category : 1
});

db.test.insert({
    ts : 60,
    category : 2
});

db.test.insert({
    ts : 40,
    category : 3
});

So when sorting it by ts all looks good, but when I use both $sort and $group result goes in a wrong order. Query:

db.test.aggregate([
{
    $sort : {ts: 1}
},
{
    $group:{"_id":"$category"}
}
]);

And the result in reverse order:

{ "_id" : 1 }
{ "_id" : 2 }
{ "_id" : 3 }

Is it Mongo feature or my misunderstanding? Maby mongo firstly applied grouping and then can't sort by absent field. For this reason probably mongoose prohibits use distinct with sorting.

Answer 1

You need to first $group and $sort the result. Since you only want the _id field you will need the $project stage.

db.test.aggregate(
    [
        { "$group": { "_id": "$category" }},
        { "$sort" : { "ts": 1 }},
        { "$project": { "_id": 1 }}
    ]
);

Answer 2

If you want to sort the other way, do it like this:

db.test.aggregate([
{
    $sort : {ts: -1}
},
{
    $group:{"_id":"$category"}
}
]);

Notice the - in front of the 1.

Answer 3

When you first $sort by ts, you are basically sorting all the elements from your collection. Thus, if you were to only run the $sort stage in the aggregation pipeline, you would get the following result:

//Query
db.test.aggregate([
    { $sort: { ts: 1} }
]);

//Output
{ "_id" : ObjectId("55141da6e4c260ae9e00832b"), "ts" : 40, "category" : 3 }
{ "_id" : ObjectId("55141d9fe4c260ae9e00832a"), "ts" : 60, "category" : 2 }
{ "_id" : ObjectId("55141d99e4c260ae9e008329"), "ts" : 80, "category" : 1 }
{ "_id" : ObjectId("55141d93e4c260ae9e008328"), "ts" : 100, "category" : 1 }

In your code, when you add the $group stage, you are basically grouping the above results by the category field, producing the output that you get:

{ "_id" : 1 }
{ "_id" : 2 }
{ "_id" : 3 }

In the end it all depends on what you are trying to achieve.

If you want to return the categories filtered by the ts field, you should only use the $sort stage and then manipulate the resulting data set:

var data = db.test.aggregate([
                    {$sort: { ts: 1}}, 
                    {$project: { 
                        _id: 0, 
                        ts: 1, 
                        category: 1
                        }
                    }
]).toArray();

for(var i = 0; i < data.length; i++) {
    console.log(data[i].category); //Output 3, 2, 1 in that sequence, on different lines
}