How to wait on multiple blocking queues in parallel?

问题

I have two separated blocking queues. The clients usually use either the first of the second blocking queue to retrieve elements to be processed.

In some case, the clients are interested in elements from the two blocking queues, whichever queue provides the data first.

How can a client wait for the two queues, in parallel?

回答1:

You could try using the poll method in some sort of loop to only wait a specified amount of time for one queue before polling the other one.

Other than that, I'd say running the blocking operations for each queue on separate threads and providing a callback interface to your main application is another, slightly more complex, option.

回答2:

I would create some sort of aggregated queue that will delegate operations to real queues:

class AggregatedQueue<E> /*implements Queue<E>*/ {
    final List<BlockingQueue<E>> queues;
    final ExecutorService service;

    final AtomicBoolean shutdown = new AtomicBoolean(false);
    final BlockingQueue<E> aggregatedQueue = new LinkedBlockingQueue<>();

    public AggregatedQueue(List<BlockingQueue<E>> queues) {
        this.queues = queues;
        this.service = Executors.newFixedThreadPool(queues.size());
    }

    public void start() {
        for (BlockingQueue<E> queue : queues) {
            service.submit((Runnable) () -> {
                while (!shutdown.get()) {
                    try {
                        aggregatedQueue.add(queue.take());
                    } catch (InterruptedException e) {
                        // handle
                    }
                }
            });
        }
    }

    //@Override
    public E take() throws InterruptedException {
        return aggregatedQueue.take();
    }
}

To use it, you should pass your queues in constructor and call start method. There would be 1 thread listening for each queue and passing taken element to aggregated queue.

AggregatedQueue<Message> queue = new AggregatedQueue<>(Arrays.asList(q1, q2));
queue.start();
while (listening) {
    Message msg = queue.take();
    dispatch(msg);
}
queue.shutdown();

回答3:

I came across the same problem, and ended up writing my own concurrent queue to support this usage pattern.

As Java blocking primitived don't allow to block on more than one object, the solution was pushed to the collection itself: Linked Blocking Multi Queue is actually a set set of queues connected by the heads. Elements are offered to individual "sub-queues", but retrieved from a single place (which suppors blocking operations).

来源：https://stackoverflow.com/questions/36330623/how-to-wait-on-multiple-blocking-queues-in-parallel

标签

java

concurrency

blockingqueue