Cache/Re-Use a Subquery in MySQL

Question

I have a very complex MySQL query that includes use of the same subquery three times. Will MySQL actually run the subquery three times? (It\'s an expensive one.) If so, is t

Answer 1

See what EXPLAIN EXTENDED says.

If it says DEPENDENT SUBQUERY or UNCACHEABLE SUBQUERY, then it will be reevaluated each time it's used.

This happens if the subquery uses session variables or is a correlated subquery.

If it doesn't, it most probably will be cached.

If your case the subquery will not be cached, it will be reevaluated in each UNION'ed set.

You subquery, though, seems to be too complicated. Why don't you just use:

SELECT id
FROM   playlist_program_map ppm, programs p
WHERE  ppm.playlist_id = 181
       AND p.id = ppm.program_id
       AND submitter_id = 32
       AND feed_id = 2478

If you have an index on playlist_program_map (playlist_id), this query should work like a charm.

Could you please tell me two more things:

1. How many rows are there in playlist_program_map and how many DISTINCT playlist_id values are there?
   • How many rows are there in programs and how many DISTINCT submitter_id, feed_id pairs are there?

From your comment I can conclude that there are 10 programs per playlist in average, and 200 programs per (submitter, feed) pair. This means your index on playlist_program_map is more selective than the one on (submitter, feed), and playlist_program_map must be leading in the join.

The fulltext index in your case also doesn't seem to be very selective, given that you need to join 10 programs out of 2,000,000.

You may better try the following:

SELECT object_id, programs.created AS created
FROM   playlist_program_map ppm, programs p, comments_programs cp
WHERE  ppm.playlist_id = 181
       AND p.id = ppm.program_id
       AND p.submitter_id = 32
       AND p.feed_id = 2478
       AND cp.object_id = p.id
       AND cp.text REGEXP 'excellent'

, and repeat this for all three tables.

Answer 2

For whatever reason, mysql IN clauses with a sub-select run very slow. Better to use join. Your sub-query becomes:

SELECT id from programs P1 INNER JOIN programs P2 ON P1.id = P2.id INNER JOIN playlist_program_map PMAP ON P2.id = PMAP.program_id WHERE P1.submitter_id=32 AND P2.feed_id=2478 AND PMAP.playlist_id=181

It will run much faster.