NEST Search whole document C# Elasticsearch

问题

I want to make a query over a million documents in Elasticsearch using Nest. My code:

var response = client.Search<MyObject>(s => s
          .Index("test")
          .Type("one")
          .Query(q => q.
                Term(
                    t => t.name, "A"
                )
          )
          .Size(10000)
          .Scroll("10m")
          .Pretty()
        );

My MyObject class:

public class MyObject
    {
        public int id { get; set; }
        public int age { get; set; }
        public string lastname { get; set; }
        public string name { get; set; }
    }

The problem is when this query is not found in the first 10k documents, it won't continue searching the rest of the results scroll API.

My question is how to achieve this (i.e moving through the whole pages in Scroll API despite there is no hits..)?

回答1:

The query will search all documents, but will only return you the top .Size number of documents.

You can paginate results using .From() and .Size(), however, deep pagination is likely a concern when paginating over a million documents. For this, you would be better to use the scroll API to efficiently retrieve 1 million documents. NEST has an observable helper ScrollAll() to help with this

var client = new ElasticClient();

// number of slices in slice scroll
var numberOfSlices = 4;

var scrollObserver = client.ScrollAll<MyObject>("1m", numberOfSlices, s => s
    .MaxDegreeOfParallelism(numberOfSlices)
    .Search(search => search
        .Index("test")
        .Type("one")
        .Term(t => t.name, "A")
    )
).Wait(TimeSpan.FromMinutes(60), r =>
{
    // do something with documents from a given response.
    var documents = r.SearchResponse.Documents;
});

来源：https://stackoverflow.com/questions/48764908/nest-search-whole-document-c-sharp-elasticsearch

标签

ElasticSearch

nest