message-queue

I've been playing with message queues (System V, but POSIX should be ok too) in Linux recently and they seem perfect for my application, but after reading The Art of Unix Programming I'm not sure if they are really a good choice. http://www.faqs.org/docs/artu/ch07s02.html#id2922148 The upper, message-passing layer of System V IPC has largely fallen out of use. The lower layer, which consists of shared memory and semaphores, still has significant applications under circumstances in which one needs to do mutual-exclusion locking and some global data sharing among processes running on the same

I'm processing messages from a Kafka topic with Samza. Some of the messages come with a timestamp in the future and I'd like to postpone the processing until after that timestamp. In the meantime, I'd like to keep processing other incoming messages. What I tried to do is make my Task queue the messages and implement the WindowableTask to periodically check the messages if their timestamp allows to process them. The basic idea looks like this: public class MyTask implements StreamTask, WindowableTask { private HashSet<MyMessage> waitingMessages = new HashSet<>(); @Override public void process

I was just seeing the documentation of three methods which can be used to execute a piece of code in the UI thread while we are working in a worker thread. The methods are: public final void runOnUIThread(Runnable action) - Runs the specified action on the UI thread. If the current thread is the UI thread, then the action is executed immediately. If the current thread is not the UI thread, the action is posted to the event queue of the UI thread public boolean post(Runnable action) - Causes the Runnable to be added to the message queue. The runnable will be run on the user interface thread.

Our application is getting a System Call Failed RPC error from DCOM ( 0x80010100 ), we suspect that the target thread's message queue is full (although I'm not convinced this is ture). I know the queue is limited to 10,000 messages and I want to see if we're close to this number in the common cases. Is there a way to monitor the size of a thread's message queue? The most promising option I found was GetQueueStatus but this doesn't include the number of messages in the queue only their types. I don't know of any way of monitoring the size of the queue (short of using a kernel debugger and

I'm looking for a Python class (preferably part of the standard language, rather than a 3rd party library) to manage asynchronous 'broadcast style' messaging. I will have one thread which puts messages on the queue (the 'putMessageOnQueue' method must not block) and then multiple other threads which will all be waiting for messages, having presumably called some blocking 'waitForMessage' function. When a message is placed on the queue I want each of the waiting threads to get its own copy of the message. I've looked at the built-in Queue class, but I don't think this is suitable because

I'm replacing some homegrown code with Celery, but having a hard time replicating the current behaviour. My desired behaviour is as follows: When creating a new user, a message should be published to the tasks exchange with the user.created routing key. Two Celery tasks should be trigged by this message, namely send_user_activate_email and check_spam . I tried implementing this by defining a user_created task with a ignore_result=True argument, plus a task for send_user_activate_email and check_spam . In my configuration, I added the following routes and queues definitions. While the message

I'm having trouble keeping my app responsive to user actions. Therefore, I'd like to split message processing between multiple threads. Can I simply create several threads, reading from the same message queue in all of them, and letting which ever one is able process each message? If so, how can this be accomplished? If not, can you suggest another way of resolving this problem? You cannot have more than one thread which interacts with the message pump or any UI elements. That way lies madness. If there are long processing tasks which can be farmed out to worker threads, you can do it that way

I was able to consume multiple messages that are sent by multiple producers to the same exchange with different routing key using the above code and was able to insert each message to database. But this will consume too much of resources as messages will be inserted into DB one after the other. So I decided to go for batch insert and I found I can set BasicQos After setting the message limit to 10 in BasicQos, my expectation is the Console.WriteLine must write 10 messages, but it is not as expected. My expectation is to consume N number messages from the queue and do bulk insert and on