How to add shards dynamically to collection in solr?

末鹿安然 提交于 2019-12-06 10:31:19

Now it's possible with Solr 4.4.0 (feature introduced in 4.3):

https://cwiki.apache.org/confluence/display/solr/Collections+API#CollectionsAPI-SplitaShard

One solution to the problem is to use the "implicit router" when creating your Collection.

Solr does supports the ability to add New Shards (or DELETE existing shards) to your index (whenever you want) via the "implicit router" configuration (CREATE COLLECTION API).

Lets say - you have to index all "Audit Trail" data of your application into Solr. New Data gets added every day. You might most probably want to shard by year.

You could do something like the below during the initial setup of your collection:

admin/collections?
action=CREATE&
name=AuditTrailIndex&
router.name=implicit&
shards=2010,2011,2012,2013,2014&
router.field=year

The above command: a) Creates 5 shards - one each for the current and the last 4 years 2010,2011,2012,2013,2014 b) Routes data to the correct shard based on the value of the "year" field (specified as router.field)

In December 2014, you might add a new shard in preparation for 2015 using the CREATESHARD API (part of the Collections API) - Do something like:

/admin/collections?
action=CREATESHARD&
shard=2015&
collection=AuditTrailIndex

The above command creates a new shard on the same collection.

When its 2015, all data will get automatically indexed into the "2015" shard assuming your data has the "year" field populated correctly to 2015.

In 2015, if you think you don't need the 2010 shard (based on your data retention requirements) - you could always use the DELETESHARD API to do so:

/admin/collections?
action=DELETESHARD&
shard=2015&
collection=AuditTrailIndex

P.S. This solution only works if you used the "implicit router" when creating your collection. Does NOT work when you use the default "compositeId router" - i.e. collections created with the numshards parameter.

This feature is truly a gamechanger - allows shards to be added dynamically based on growing demands of your business.

Is this feature available in Elastic Search. If not, I am sure they will in time.

Currently you cannot add new shards once the collection is made

https://issues.apache.org/jira/browse/SOLR-3755

易学教程内所有资源均来自网络或用户发布的内容,如有违反法律规定的内容欢迎反馈
该文章没有解决你所遇到的问题?点击提问,说说你的问题,让更多的人一起探讨吧!