Rabbitmq- Designing a message replay service

Question

I am trying to design a replay mechanism that will enable users to replay messages from the queues. The best design I have come up for an exchange that contains multiple queues

Answer 1

The first solution is feasible. Given that rabbit MQ ships with a tracing plugin, storing the message in DB becomes even more easier as it simply boils down to consuming from a queue bound to amq.rabbitmq.trace exchange.

This exchange is specific to a vhost and every vhost has its own amq.rabbitmq.trace exchange. In addition when creating a new trace it is possible to restrict on which queue/exchange to enable tracing , therby giving the flexibility to disable tracing where it is not required.

Refer the following links to configure tracing:
https://www.rabbitmq.com/firehose.html
https://www.rabbitmq.com/blog/2011/09/09/rabbitmq-tracing-a-ui-for-the-firehose/

Answer 2

1. Does that make sense?

Yes

2. Am I inventing the wheel? Is there a rabbit inherent solution? plugin?

You are not reinventing the wheel. There is AFAIK no rabbit solution and no out of the box solution.

Your first solution is in my opinion good. The Another solution is very problematic, because a healthy rabbit is an empty one and rabbit is not a datastore.

You will have a queue (STORE) where all published messages should be routed to. Instead of binding STORE with all the binding keys, you could consider using a topic exchange. At the price that binding key must not contain . # * and a slight overhead when the message is routed. The STORE queue will bind with the binding key #.

You could have a look at firehose.

Why a web request? You can use Rabbit with an RPC call:

• Subscriber sends an amqp request to start replay with a filter ( startdate, etc).
• Request contains the reply-to queue name. The queue may be exclusive to the client and request.
• RPC server pulls data from the DB and publishes it to the reply-to queue

You could also have a look at the direct-reply-to pattern.

3. Does creating multiple exchanges considered a good practice?

Yes, as soon as you need it. For your specific case, in my opinion an exchange per subscriber is not necessary. The request contains already the queue name. You could simply publish to this queue using the default exchange amq.direct with the routing key equal to the queue name. If you want an exchange I would create an unique exchange (eg. "replay") and have each subscriber bind their replay queues to this exchange. The "replay" exchange can be a convention or sent with the request.