Processing large amounts of data in PHP without a browser timeout

Question

I have array of mobile numbers, around 50,000. I\'m trying to process and send bulk SMS to these numbers using third-party API, but the browser will freeze for some minutes

Answer 1

If this isn't a one-off type of situation, consider engineering a better solution.

What you basically want is a queue that your browser-bound process can write to, and than 1-N worker processes can read from and update.

Putting work in the queue should be rather inexpensive - perhaps a bunch of simple INSERT statements to a SQL RDBMS.

Then you can have a daemon or two (or 100, distributed across multiple servers) that read from the queue and process stuff. You'll want to be careful here and avoid two workers taking on the same task, but that's not hard to code around.

So your browser-bound workflow is: click some button that causes a bunch of stuff to get added to the queue, then redirect to some "queue status" interface, where the user can watch the system chew through all their work.

A system like this is nice, because it's easy to scale horizontally quite a ways.

EDIT: Christian Sciberras' answer is going in this direction, except the browser ends up driving both sides (it adds to the queue, then drives the worker process)