MySQL: Alternatives to ORDER BY RAND()

Question

I\'ve read about a few alternatives to MySQL\'s ORDER BY RAND() function, but most of the alternatives apply only to where on a single random result is needed.

Answer 1

The solution I am using is also posted in the link below: How can i optimize MySQL's ORDER BY RAND() function?

I am assuming your users table is going to be larger than your profiles table, if not then it's 1 to 1 cardinality.

If so, I would first do a random selection on user table before joining with profile table.

First do selection:

SELECT *
FROM users
WHERE users.ownership = 1 OR users.stamp = 1

Then from this pool, pick out random rows through calculated probability. If your table has M rows and you want to pick out N random rows, the probability of random selection should be N/M. Hence:

SELECT *
FROM
(
    SELECT *
    FROM users
    WHERE users.ownership = 1 OR users.stamp = 1
) as U
WHERE 
    rand() <= $limitCount / (SELECT count(*) FROM users WHERE users.ownership = 1 OR users.stamp = 1)

Where N is $limitCount and M is the subquery that calculates the table row count. However, since we are working on probability, it is possible to have LESS than $limitCount of rows returned. Therefore we should multiply N by a factor to increase the random pool size.

i.e:

SELECT*
FROM
(
    SELECT *
    FROM users
    WHERE users.ownership = 1 OR users.stamp = 1
) as U
WHERE 
    rand() <= $limitCount * $factor / (SELECT count(*) FROM users WHERE users.ownership = 1 OR users.stamp = 1)

I usually set $factor = 2. You can set the factor to a lower value to further reduce the random pool size (e.g. 1.5).

At this point, we would have already limited a M size table down to roughly 2N size. From here we can do a JOIN then LIMIT.

SELECT * 
FROM
(
       SELECT *
        FROM
        (
            SELECT *
            FROM users
            WHERE users.ownership = 1 OR users.stamp = 1
        ) as U
        WHERE 
            rand() <= $limitCount * $factor / (SELECT count(*) FROM users WHERE users.ownership = 1 OR users.stamp = 1)
) as randUser
JOIN profiles
ON randUser.id = profiles.memberid AND profiles.photo != ''
LIMIT $limitCount

On a large table, this query will outperform a normal ORDER by RAND() query.

Hope this helps!