Is it possible to iterate through documents stored in Lucene Index?

Question

I have some documents stored in a Lucene index with a docId field. I want to get all docIds stored in the index. There is also a problem. Number of documents is about 300 00

Answer 1

If you use .document(i) as in above examples and skip over deleted documents be careful if you use this method for paginating results. i.e.: You have a 10 docs/per page list and you need to get the docs. for page 6. Your input might be something like this: offset=60,count = 10 (documents from 60 to 70).

    IndexReader reader = // create IndexReader
for (int i=offset; i

You will have some problems with the deleted ones because you should not start from offset=60, but from offset=60 + the number of deleted documents that appear before 60.

An alternative I found is something like this:

    is = getIndexSearcher(); //new IndexSearcher(indexReader)
    //get all results without any conditions attached. 
    Term term = new Term([[any mandatory field name]], "*");
    Query query = new WildcardQuery(term);

    topCollector = TopScoreDocCollector.create([[int max hits to get]], true);
    is.search(query, topCollector);

   TopDocs topDocs = topCollector.topDocs(offset, count);

note: replace text between [[ ]] with own values. Ran this on large index with 1.5million entries and got random 10 results in less than a second. Agree is slower but at least you can ignore deleted documents if you need pagination.