ElasticSearch usage with MySQL

Question

I\'m using ElasticSearch for the search component of a site. The data that is being indexed and eventually searched is the same data that is being saved in a MySQL DB.

Answer 1

There is no "right way" to use Elasticsearch. "Right" is relative, so the "right way" is a way that supports your use case(s). Elasticsearch doesn't only work for one specific use case, but for increasingly many more than one use case.

The case you describe is a perfectly valid one, i.e. indexing in ES whatever content you have in another RDBMS such as MySQL and making sure the indexed content is in synch with the primary source of truth.

One difficult thing in your use case that you need to keep in mind is that you have to guarantee that MySQL and ES are always 1:1 in synch, and that's not necessarily easy to do for various reasons:

• what happens if you need to bring ES down for maintenance, but your app has to stay up for whatever reason?
• what happens if there's an issue in ES and a document doesn't get indexed/updated/deleted? (remember there's no transactional support)

There are otherways to synch MySQL and ES that are less brittle, e.g. by using the binlog.

You need to ask yourself those questions and figure out a strategy to mitigate those potential problems, because I can assure you they (and others) will definitely arise.

To sum up, there's no problem with your architecture, thousands of companies do the exact same thing, however, you need to have a plan if your synch plan goes south.

Answer 2

ElasticSearch does not fit very well for updating/deleting documents in large scale.

There are many aproaches that try to minimize the overloading about this downside on it's architecture, but if think that increases the complexity of your solution.

I suggest that you keep CRUD operations only on MySQL and use ES as append-only. Actually, StackOverflow itself, and many others great TI companies uses this approach.