beanstalkd

I have set up queues in Laravel for my processing scripts. I am using beanstalkd and supervisord. There are 6 different tubes for different types of processing. The issue is that for each tube, artisan is constantly spawning workers every second. The worker code seems to sleep for 1 second and then the worker thread uses 7-15% cpu, multiply this by 6 tubes... and I would like to have multiple workers per tube.. my cpu is being eaten up. I tried changing the 1 second sleep to 10 seconds. This helps but there is still a huge cpu spike every 10 seconds when the workers wake back up. I am not even

I have beanstalk installed on my server, and run the following: $ telnet localhost 11300 use my_tube USING my_tube peek-ready FOUND 11065 41 {"ts":1295537419,"data":{"nid":"212156"}} Now my question is: how can I get a list of all jobs in the queue; something like peek-all ? There are some forks of the main code that provide this sort of information ( retrieving ready, reserved, and buried jobs in a tube ), but its not in the main trunk version. This is because of the problems that it can bring - such as on a huge implementation it would have to return a potentially vast amount of data and

Here is my needs: Enqueue_in(10.hours, ... ) (DJ syntax is perfect.) Multiply workers, concurrently. (Resque or beanstalkd are good for this, but not DJ) Must handle push and pop of 100 jobs a second. (I will need to run a test to make sure, but I think DJ can't handle this many jobs) Resque and beanstalkd don't do the enqueue_in. There is a plugin (resque_scheduler) that does it, but I'm not sure of how stable it is. Our enviroment is on amazon, and they rolled out the beanstalkd for free for who has amazon instances, that is a plus for us, but I'm still not sure what is the best option here.

The following code is a snipet taken from an example of pheanstalk being implemented and working properly (obtained from pheanstalk's github page: https://github.com/pda/pheanstalk ): <?php require_once("vendor/autoload.php"); use Pheanstalk\Pheanstalk; $pheanstalk = new Pheanstalk('127.0.0.1'); // ------------ producer (queues jobs): $pheanstalk ->useTube('testtube') ->put("job payload goes here\n"); // ------------ worker (performs jobs): $job = $pheanstalk ->watch('testtube') ->ignore('default') ->reserve(); echo $job->getData(); $pheanstalk->delete($job); // ------------ check server