Fastest way to generate 11,000,000 unique ids

问题

I am trying to create php script that generates 11,000,000 million unique ids in sequential order. However, I am trying to do it very quickly within 20 min it should generate these 11 million unique ids. Also, once it reaches 12,000,000 it should wrap around and start back at zero.

Here is what I have so far. This script would would only return one id at a time. I just added a loop to see how long it would take to generate the ids.

while(true){

    try {

        $this->getAdapter()->query('INSERT INTO generate_ids (assigned_id) SELECT (MAX(assigned_id)+1) FROM generate_ids');
        $id = $this->getAdapter()->lastInsertId();


        $sql = 'SELECT assigned_id FROM generate_ids WHERE id = $id';  
        $query = $this->getAdapter()->query($sql);
        $result = $query->fetchAll();
            //Live system would return id here
        $assigned_id = $result[0]['assigned_id'];


    } catch (Exception $e) {
        //do nothing
    }

    if($count == 11000000){
        die();
    }

    $count++;
}

}

回答1:

If you create the following table:

 CREATE TABLE sequence (
     sequence_id BIGINT NOT NULL AUTO_INCREMENT,
     PRIMARY KEY (`sequence_id`)
)

Then issue these three queries one after the other:

INSERT INTO sequence () VALUES ();
DELETE FROM sequence WHERE sequence_id < LAST_INSERT_ID();
SELECT LAST_INSERT_ID() AS sequence;

The third query is guaranteed to return a unique sequence number. This guarantee holds even if you have dozens of different client programs connected to your database. That's the beauty of AUTO_INCREMENT.

Instead of just generating eleven million of these sequence numbers up front, you can use these SQL queries to get a unique sequence number whenever you need it.

If you must wrap around at sequence number 12 million you can use these queries instead.

INSERT INTO sequence () VALUES ();
DELETE FROM sequence WHERE sequence_id < LAST_INSERT_ID();
SELECT LAST_INSERT_ID() MOD 12000000 AS sequence;

The trick here is to use an auto-increment sequence number for uniqueness, but to also delete the rows in the table so it doesn't gobble up lots of space.

Note that you can also use the sequence number of LAST_INSERT_ID() for other purposes, like so for example.

INSERT INTO sequence () VALUES ();
DELETE FROM sequence WHERE sequence_id < LAST_INSERT_ID();
INSERT INTO user (userid, username, phone) 
          VALUES (LAST_INSERT_ID() MOD 12000000, 'Joe', '800-555-1212');
SELECT LAST_INSERT_ID() MOD 12000000 AS sequence;

回答2:

If you need to insert all your ids at once (for some reason) the fastest approach would be to do it in pure SQL

insert into generate_ids (assigned_id)
select N
from
(
select a.N + b.N * 10 + c.N * 100 + d.N * 1000 + e.N * 10000 + f.N * 100000 + g.N * 1000000 + h.N * 10000000 + 1 as N
from (select 0 as N union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9) a
    ,(select 0 as N union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9) b
    ,(select 0 as N union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9) c
    ,(select 0 as N union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9) d
    ,(select 0 as N union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9) e
    ,(select 0 as N union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9) f
    ,(select 0 as N union all select 1 union all select 2 union all select 3 union all select 4 union all select 5 union all select 6 union all select 7 union all select 8 union all select 9) g
    ,(select 0 as N union all select 1) h
order by N
) q
 where N <= 11000000

It takes less than a minute to complete on my laptop.

来源：https://stackoverflow.com/questions/20110896/fastest-way-to-generate-11-000-000-unique-ids

标签

php

mysql

scripting