Redis Vs RabbitMQ as a data broker/messaging system in between Logstash and elasticsearch

Question

We are defining an architecture to collect log information by Logstash shippers which are installed in various machines and index the data in one elasticsearch server centra

Answer 1

Quick questions to ask:

1. why do you need a broker? If you're using logstash or logstash-forwarder to read files from these servers, they both will slow down if the pipeline gets congested.
2. do you have any experience with administering rabbit or redis? All things being equal, the tool you know how to use is the better tool.

In the realm of opinions, I've run redis as a broker, and hated it. Of course, that could have been my inexperience with redis (not a problem with the product itself), but it was the weakest link in the pipeline and always failed when we needed it most.