Indexing every column in a table

Question

I have a couple of questions regarding MySQL indexing:

1) Is there any speed increase when indexing a table stored in memory?

2) When searching my table I ma

Answer 1

Re Q1... The query Optimizer sometimes chooses to scan the table even when there is a "perfectly good" index. This tradeoff is based on a complex algorithm, but, as a Rule of Thumb:

If more than ~20% of the index needs to be used, it is deemed more efficient to ignore the index and simply scan the table.

The reasoning for this is: Using an index means scanning the index BTree (which looks very much like a table), then jumping over to the data BTree to find the record. This back-and-forth is avoided if it simply scans the data. The drawback is that it needs to ignore up to 80% of the rows.

Corollary: Don't bother indexing "flags" (0/1, T/F, M/F, Yes/No) or low cardinality columns (yes/no/maybe, M/F/etc, day-of-week, ...).

On the other hand, it may be very useful to have a composite index starting with a low cardinality column:

WHERE deleted=0 AND created_at > NOW() - INTERVAL 1 DAY
INDEX(deleted, created_at)