How to update bigger solr index efficiently

和自甴很熟 提交于 2020-01-03 06:21:09

问题


I have a larger Solr index size. The index contains 3 million documents. I need to update some documents frequently. But each time Solr commit takes some time since the index contains millions of documents. And each commit opens a new searcher. So it takes some time to perform search for first time. Is there any way to update the documents in an efficient manner? Thanks in advance.


回答1:


You can check for Solr 4 soft commits which would make the indexing faster. The new documents would be available as soon as committed but would not be persisted to index. You can decide to perform hard commits once in a while.

Documentation

A soft commit is much faster since it only makes index changes visible and does not fsync index files or write a new index descriptor. If the JVM crashes or there is a loss of power, changes that occurred after the last hard commit will be lost. Search collections that have near-real-time requirements (that want index changes to be quickly visible to searches) will want to soft commit often but hard commit less frequently.

Recovering can probably be performed with the transaction log.

If losing documents is an issue to you, you can configure Master Slave architecture for Solr. Master is where the indexing happens and slaves where the search is performed, so that the indexing and commit times do not impact the search much. However, there would be latency and as well as depends how frequents your commits are.



来源:https://stackoverflow.com/questions/17058969/how-to-update-bigger-solr-index-efficiently

标签
易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!