Mysql performance on 6 million row table

Question

One day I suspect I\'ll have to learn hadoop and transfer all this data to a non-structured database, but I\'m surprised to find the performance degrade so significantly in

Answer 1

I would imagine that a 6M row table should be able to be optimised with quite normal techniques.

I assume that you have a dedicated database server, and it has a sensible amount of ram (say 8G minimum).

You will want to ensure you've tuned mysql to use your ram efficiently. If you're running a 32-bit OS, don't. If you are using MyISAM, tune your key buffer to use a signficiant proportion, but not too much, of your ram.

In any case you want to run repeated performance testing on production-grade hardware.