Redis publish-subscribe: Is Redis guaranteed to deliver the message even under massive stress?

Question

Provided that both the client subscribed and the server publishing the message retain the connection, is Redis guaranteed to always deliver the published message to the subs

Answer 1

Redis does not provide guaranteed delivery using its Pub/Sub mechanism. Moreover, if a subscriber is not actively listening on a channel, it will not receive messages that would have been published.

I previously wrote a detailed article that describes how one can use Redis lists in combination with BLPOP to implement reliable multicast pub/sub delivery:

http://blog.radiant3.ca/2013/01/03/reliable-delivery-message-queues-with-redis/

For the record, here's the high-level strategy:

• When each consumer starts up and gets ready to consume messages, it registers by adding itself to a Set representing all consumers registered on a queue.
• When a producers publishes a message on a queue, it:
  • Saves the content of the message in a Redis key
  • Iterates over the set of consumers registered on the queue, and pushes the message ID in a List for each of the registered consumers
• Each consumer continuously looks out for a new entry in its consumer-specific list and when one comes in, removes the entry (using a BLPOP operation), handles the message and moves on to the next message.

I have also made a Java implementation of these principles available open-source: https://github.com/davidmarquis/redisq

These principles have been used to process about 1,000 messages per second from a single Redis instance and two instances of the consumer application, each instance consuming messages with 5 threads.