In my web site I have some feature to upload data feeds by users. These data feeds normally having more than 30,000 records. And each records need to be go through complex v
PHP is not designed for this kind of job. But there are tricks to try doing it.
If your feeds have more than 30,000 records each, you will have timeout problems with PHP FPM. So your script must run with PHP CLI (Check your server parameters for PHP CLI timeout).
You should look at Rabbit MQ and program a cron job each minute to consume
messages in the queue. Here an example with PHP http://www.rabbitmq.com/tutorials/tutorial-two-php.html
Hope that will help you