Optimizing my mysql statement! - RAND() TOO SLOW

Question

So I have a table with over 80,000 records, this one is called system. I also have another table called follows.

I need my statement to randomly select records from

Answer 1

Perhaps a little late, but at least here is an extra solution for future consideration:

SELECT minSystem.id, 
    minSystem.username, 
    minSystem.password, 
    minSystem.followed, 
    minSystem.isvalid,
    randFollows.userid, 
    randFollows.systemid
FROM
(
    SELECT *
    FROM system
    WHERE system.followed = 0 AND system.isvalid = 1
) as minSystem
LEFT JOIN 
(
    SELECT * 
    FROM (
        SELECT *
        FROM follows
        WHERE follows.systemid IS NULL
    ) as minFollows
    WHERE rand() <= 200 * 1.5 / (SELECT count(*) FROM follows WHERE systemid IS NULL)
) as randFollows
ON minSystem.id = randFollows.systemid
LIMIT 200

First, we perform a selection on the system table to cut down the minSystem and minFollow temp table size. Then we select random rows from the minFollows table through calculated probability. By now we will have a fairly random randFollows table to LEFT JOIN with minSystem. Finally we do LIMIT 200.

If you are using MyISam, you can simply retrieve the table size. This eliminates the extra subquery to calculate the follows table size. Alternatively, you can also hardcode the denominator if your table size doesn't grow too fast (This requires more manual maintenance however).

For more thorough explaination, please checkout the solution I posted on: MySQL: Alternatives to ORDER BY RAND()

Hope this helps (or at least I hope you'll find this interesting)!