Change dynamically elasticsearch synonyms

旧巷老猫 提交于 2019-12-29 03:38:07

问题


Is it possible to store the synonyms for elasticsearch in the index? Or is it possible to get the synonym list from a database like couchdb? I'd like to add synonyms dynamically to elasticsearch via the REST-API.


回答1:


There are two approaches when working with synonyms :

  • expanding them at indexing time,
  • expanding them at query time.

Expanding synonyms at query time is not recommended since it raises issues with :

  • scoring, since synonyms have different document frequencies,
  • multi-token synonyms, since the query parser splits on whitespaces.

More details on this at http://wiki.apache.org/solr/AnalyzersTokenizersTokenFilters#solr.SynonymFilterFactory (on Solr wiki, but relevant for ElasticSearch too).

So the recommended approach is to expand synonyms at indexing time. In your case, if the synonym list is managed dynamically, it means that you should re-index every document which contains a term whose synonym list has been updated so that scoring remains consistent between documents analyzed pre and post update. I'm not saying that it is not possible but it requires some work and will probably raise performance issues with synonyms which have a high frequency in your index.




回答2:


There are few new solutions now to those proposed in other answers few years ago. The two main approaches implemented as plugins:

  1. The file-watcher-synonym filter is a plugin that can periodically reload synonyms every given numbers of seconds, as defined by user.
  2. The refresh-token-plugin allows a real-time update of the index. However, this plugin aparrently has some problems, which stem from the fact that elasticsearch is unable to distinguish between analyzers at search time only from those used at index time.

Good discussion on this subject can be found on the elastisearch github ticket system: https://github.com/brusic/refresh-token-filters




回答3:


It isn't too painful in elasticsearch to update the synonym list. It can be done by opening and closing You could have it driven from anywhere, but need some of your own infrastructure. It'd work like this:

  • You want an alias pointing at your current index
  • Sync down a new index file to your servers
  • Create a new index with a custom analyzer that uses the new index
  • Rebuild the content from current index to new index
  • Repoint index alias from current to new index


来源:https://stackoverflow.com/questions/7272368/change-dynamically-elasticsearch-synonyms

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!