message-queue

Here's a theoretical question: When I'm building an application using message queueing, I'm going to need multiple queues support different data types for different purposes. Let's assume I have 20 queues (e.g. one to create new users, one to process new orders, one to edit user settings, etc.). I'm going to deploy this to Windows Azure using the 'minimum' of 1 web role and 1 worker role. How does one read from all those 20 queues in a proper way? This is what I had in mind, but I have little or no real-world practical experience with this: Create a class that spawns 20 threads in the worker

I try to send integer by msg queue but the function mq_send(mq, &val , sizeof(val), 0); is working for only char type pointer so is there any way to send integer to queue with another function or same function. Regards... sdg Do not read the char* in this case as the only allowed datatype. Many *ix API use char as a generic buffer pointer. View the interface therefore as taking a pointer to buffer and the size of the buffer. That buffer can be anything you like, from a single int, to a struct, seralized string representation of your class, or just about anything else in memory. int i; mq_send

I'm running ActiveMQ in a production system. Some of our queues are very high volume and some are very low volume. I'm interested in mirroring one of the low volume queues so that I can build informal monitoring services around the messages being received. Unfortunately, the only documentation I've been able to find seems to imply that Mirrored Queues are are all-or-nothing: you either create a topic for every single queue you have (and suffer the performance penalty of copying every message flowing through your system), or you can't use the feature at all. Is there no way of enabling this

I am using Kafka and Zookeeper as the main components of my data pipeline, which is processing thousands of requests each second. I am using Samza as the real time data processing tool for small transformations that I need to make on the data. My problem is that one of my consumers (lets say ConsumerA ) consumes several topics from Kafka and processes them. Basically creating a summary of the topics that are digested. I further want to push this data to Kafka as a separate topic but that forms a loop on Kafka and my component. This is what bothers me, is this a desired architecture in Kafka?