message-queue

I'm working on a (primarily) Perl project and want to use a message queue to isolate processes from each other. I have a work flow like this: Input -> Receiver -> Processor(s) -> Output(s) I need to handle several hundred transactions/second, so speed my biggest motivator. What is the fastest message queue system for this type of setup? All of my processes run on the same machine, so I can use UNIX sockets or temp-files if it means better performance. So far my two favorites are IPC::DirQueue and beanstalkd. Anyone have experience with both and can make conclusions about which is faster? What

I need to implement a queue using table. The business requirement is to have a single queue which will be accessed by 5-10 boxes to get the next job/jobs. There will not be more than 5000 jobs per day. Also, a batch of jobs should be "dequeued" at one time. Just wondering what are the problem areas and issues I might run into as I havent done it before. If anyone has faced this/done this before, can you please point me to a design/sample implementation or issues that need to be taken care of. Thanks There are a lot of general purpose queuing or messaging services. Even if you want to implement

Using PHP, I have a query that goes through my DB looking for pending tasks with reminder triggers at certain times of the day. I have a cronjob that runs every 10 mins and checks the DB for any rows that has "remind_me" field set to go off within the next 10 mins. If it does find something, what's the best way to queue an email with the task information? I guess I'll need some sort of message queue system, but how does the email part work? Will I need another cronjob that runs every minute to check the queue system? If you want to do a mail queue system I would suggest you take a look at PEAR

I have a RabbitMQ queues for documents generation. Basically, each document has type and state (new, processing, ready), so I use topic exchange with routing keys like type.state . Every time document changes I send the message with last document description to the exchange and it works good enough. However sometimes document can be processed twice: User send new document. So new message report.new is sent to exchange. While worker hasn't started document processing (the queue hasn't yet reached) user updated the document. The new message report.new for the same document is sent. So now worker

I know this question has been asked a bit before. But looking around I still cant make my mind up which route I should go down. Here's my scenario, hopefully you can help out: We will have a series of web services that will be hit on a scheduled basis by hundreds of mobile applications. These services will data on the device with new information both going to the devices and coming back from them. The data returned from the devices will need to update a single central SQL server database that also feeds several desktop applications and a website. In order to reduce the amount of time for the

Do messages in dead letter queues in Azure Service Bus expire? Some explanation I have these queue settings: var queueDescription = new QueueDescription("MyTestQueue") { RequiresSession = false, DefaultMessageTimeToLive = TimeSpan.FromMinutes(1), EnableDeadLetteringOnMessageExpiration = true, MaxDeliveryCount = 10 }; namespaceManager.CreateQueue(queueDescription); When I place some messages in a Azure Service Bus message queue (not queues from Azure Storage) and don't consume them (ever), they'll be moved to the dead letter queue automatically. However, if I have no consumer for the dead

My Java application sends messages to RabbitMQ exchange, then exchange redirects messages to binded queue. I use Springframework AMQP java plugin with RabbitMQ. The problem: message comes to queue, but it stays in "Unacknowledged" state, it never becomes "Ready". What could be the reason? An Unacknowledged message implies that it has been read by your consumer, but the consumer has never sent back an ACK to the RabbitMQ broker to say that it has finished processing it. I'm not overly familiar with the Spring Framework plugin, but somewhere (for your consumer) you will be declaring your queue,

I only want to receive my message in a async method! and its freezing my UI public async void ProcessMessages() { MessageQueue MyMessageQueue = new MessageQueue(@".\private$\MyTransactionalQueue"); MyMessageQueue.Formatter = new XmlMessageFormatter(new Type[] { typeof(string) }); while (true) { MessageQueueTransaction MessageQueueTransaction = new MessageQueueTransaction(); MessageQueueTransaction.Begin(); ContainError = false; ProcessPanel.SetWaiting(); string Body = MyMessageQueue.Receive(MessageQueueTransaction).Body.ToString(); //Do some process with body string. MessageQueueTransaction