问题
What would be the impact of re-indexing the same value across multiple fields in a lucene index?
The idea is that someone's first name is a part of their name and their general details. So I would want to index that value into multiple fields. Ted Bloggs I might index as follows:
Field | Value
-------------|---------
firstName | Ted
lastName | Blogs
name | Ted
name | Bloggs
general | Ted
general | Bloggs
all | Ted
all | Bloggs
By doing this I can easily form categories of fields however I'm worried it may have adverse performance and/or disk usage impacts.
Could anyone advise please
回答1:
@aishwarya is right, but to expand on it a little bit more:
From the docs:
This file is sorted by Term. Terms are ordered first lexicographically (by UTF16 character code) by the term's field name, and within that lexicographically (by UTF16 character code) by the term's text.
The term will be stored once per field, so if you repeat each term five times your storage will be five times bigger. However, the size of the term dic is logarithmic with respect to the size of the raw data, so I doubt you will have a problem.
The performance penalty will be non-existent (Lucene caches where each field starts) except insofar as having more data will force stuff out of memory. For most search infrastructures, you'll probably have an index of under a few gb, which will easily fit in memory anyway.
回答2:
Xodarap's explanation is good, just to add some more:
The best way to think about fields in lucene is that each is its own miniature inverted index, but the document ids are aligned/parallel so you can do disjunctions/conjunctions across different fields.
one thing to be careful of when adding lots of fields: by default each field has a byte[maxdoc] loaded up in ram used for length normalization. So with many documents and lots of fields, all with length normalization enabled, this could chew up some space.
Looking at your use case length normalization is probably not very useful for fields like firstName/LastName anyway, so you may want to omitNorms() on these short fields.
回答3:
lucene's indexing is pretty well optimised so I would not worry too much about performance or disk usage here. Having said that, your use case would definitely have a poorer performance compared to a straight forward single column exact search, although not so much to cause worry.
来源：https://stackoverflow.com/questions/8356837/impact-of-repeat-value-across-multiple-fields-in-lucene