Laravel Artisan Queues - high cpu usage

Question

I have set up queues in Laravel for my processing scripts. I am using beanstalkd and supervisord. There are 6 different tubes for different types of processing.

The

Answer 1

The problem of high CPU is caused because the worker loads the complete framework everytime it checks for a job in the queue. In laravel 4.2, you can use php artisan queue:work --daemon. This will load the framework once and the checking/processing of jobs happen inside a while loop, which lets CPU breathe easy. You can find more about daemon worker in the official documentation: http://laravel.com/docs/queues#daemon-queue-worker.

However, this benefit comes with a drawback - you need special care when deploying the code and you have to take care of the database connections. Usually, long running database connections are disconnected.

Answer 2

I had the same issue.

But I found another solution. I used the artisan worker as is, but I modified the 'watch' time. By default(from laravel) this time is hardcoded to zero, I've changed this value to 600 (seconds). See the file: 'vendor/laravel/framework/src/Illuminate/Queue/BeanstalkdQueue.php' and in function 'public function pop($queue = null)'

So now the work is also listening to the queue for 10 minutes. When it does not have a job, it exits, and supervisor is restarting it. When it receives a job, it executes it after that it exists, and supervisor is restarting it.

==> No polling anymore!

notes:

• it does not work for iron.io queue's or others.
• it might not work when you want that 1 worker accept jobs from more than 1 queue.

Answer 3

According to this commit, you can now set a new option to queue:listen "--sleep={int}" which will let you fine tune how much time to wait before polling for new jobs.
Also, default has been set to 3 instead of 1.