Why does Mongo hint make a query run up to 10 times faster?

Question

If I run a mongo query from the shell with explain(), get the name of the index used and then run the same query again, but with hint() specifying the same index to be used - "millis" field from explain plan is decreased significantly

for example

no hint provided:

>>db.event.find({ "type" : "X", "active" : true, "timestamp" : { "$gte" : NumberLong("1317498259000") }, "count" : { "$gte" : 0 } }).limit(3).sort({"timestamp" : -1 }).explain();

{
    "cursor" : "BtreeCursor my_super_index",
    "nscanned" : 599,
    "nscannedObjects" : 587,
    "n" : 3,
    "millis" : 24,
    "nYields" : 0,
    "nChunkSkips" : 0,
    "isMultiKey" : true,
    "indexOnly" : false,
    "indexBounds" : { ... }
} 

hint provided:

>>db.event.find({ "type" : "X", "active" : true, "timestamp" : { "$gte" : NumberLong("1317498259000") }, "count" : { "$gte" : 0 } }).limit(3).sort({"timestamp" : -1 }).hint("my_super_index").explain();

{
    "cursor" : "BtreeCursor my_super_index",
    "nscanned" : 599,
    "nscannedObjects" : 587,
    "n" : 3,
    "millis" : 2,
    "nYields" : 0,
    "nChunkSkips" : 0,
    "isMultiKey" : true,
    "indexOnly" : false,
    "indexBounds" : { ... }
} 

The only difference is "millis" field

Does anyone know why is that?

UPDATE: "Selecting which index to use" doesn't explain it, because mongo, as far as I know, selects index for each X (100?) runs, so it should be as fast as with hint next (X-1) runs

Answer 1

Mongo uses an algorithm to determine which index to be used when no hint is provided and then caches the index used for the similar query for next 1000 calls

But whenever you explain a mongo query it will always run the index selection algorithm, thus the explain() with hint will always take less time when compared with explain() without hint.

Similar question was answered here Understanding mongo db explain

Answer 2

Mongo did the same search both times as you can see from the number of scanned objects. Also you can see that the used index was the same (take a look at the "cursor" entry), both used already your my_super_index index.

"hint" only tells Mongo to use that specific index which it already automatically did in the first query.

The second search was simple faster because all the data was probably already in the cache.

Answer 3

I struggled finding reason for same thing. I found that when we have lots of indexes, mongo is indeed taking more time than using hint. Mongo basically is taking lot of time deciding which index to use. Think of a scenario where you have 40 indexes and you do a query. First task which Mongo needs to do is which index is best suited to be used for particular query. This would imply mongo needs to scan all the keys as well as do some computation in every scan to find some performancce index if this key is used. hint will definitely speed up since index key scan will be saved.

Answer 4

I will tell you how to find out how it's faster 1) without index It will pull every document into memory to get the result 2) with index If you have a lot of index for that collection it will take index from the cache memory 3) with .hint(_index) It will take that specific index which you have mention

with hint() without hint() both time you do .explain("executionStats") with hint() then you can check totalKeysExamined value that value will match with totalDocsExamined without hint() you can see totalKeysExamined value is greter then totalDocsExamined

totalDocsExamined this result will perfectly match with result count most of the time.