Preferred method of indexing bulk data into ElasticSearch?

Question

I\'ve been looking at ElasticSearch as solution get some better search and analytics functionality at my company. All of our data is in SQL Server at the moment and I\'ve succes

Answer 1

We use RabbitMQ to pipe data from SQL Server to ES. That way Rabbit takes care of the queuing and processing.

As a note, we can run over 4000 records per second from SQL into Rabbit. We do a bit more processing before putting the data into ES but we still insert into ES at over 1000 records per second. Pretty damn impressive on both ends. Rabbit and ES are both awesome!