Streaming the result of an aggregate operation using spring-data-mongodb

问题

I am using spring-data-mongodb and I want to use a cursor for an aggregate operation.

MongoTemplate.stream() gets a Query, so I tried creating the Aggregation instance, convert it to a DbObject using Aggregation.toDbObject(), created a BasicQuery using the DbObject and then invoke the stream() method.
This returns an empty cursor.

Debugging the spring-data-mongodb code shows that MongoTemplate.stream() uses the FindOperation, which makes me thinkspring-data-mongodb does not support streaming an aggregation operation.
Has anyone been able to stream the results of an aggregate query using spring-data-mongodb?

For the record, I can do it using the Java mongodb driver, but I prefer using spring-data.

EDIT Nov 10th - adding sample code:

    MatchOperation match = Aggregation.match(Criteria.where("type").ne("AType"));
    GroupOperation group = Aggregation.group("name", "type");
    group = group.push("color").as("colors");
    group = group.push("size").as("sizes");
    TypedAggregation<MyClass> agg = Aggregation.newAggregation(MyClass.class, Arrays.asList(match, group));

    MongoConverter converter = mongoTemplate.getConverter();
    MappingContext<? extends MongoPersistentEntity<?>, MongoPersistentProperty> mappingContext = converter.getMappingContext();
    QueryMapper queryMapper = new QueryMapper(converter);
    AggregationOperationContext context = new TypeBasedAggregationOperationContext(MyClass.class, mappingContext, queryMapper);
    // create a BasicQuery to be used in the stream() method by converting the Aggregation to a DbObject
    BasicQuery query = new BasicQuery(agg.toDbObject("myClass", context));

    // spring-mongo attributes the stream() method to find() operationsm not to aggregate() operations so the stream returns an empty cursor
    CloseableIterator<MyClass> iter = mongoTemplate.stream(query, MyClass.class);

    // this is an empty cursor
    while(iter.hasNext()) {
        System.out.println(iter.next().getName());
    }

The following code, not using the stream() method, returns the expected non-empty result of the aggregation:

    AggregationResults<HashMap> result = mongoTemplate.aggregate(agg, "myClass", HashMap.class);

回答1:

For those who are still trying to find the answer to this:

From spring-data-mongo version 2.0.0.M4 onwards (AFAIK) MongoTemplate got an aggregateStream method.

So you can do the following:

 AggregationOptions aggregationOptions = Aggregation.newAggregationOptions()
        // this is very important: if you do not set the batch size, you'll get all the objects at once and you might run out of memory if the returning data set is too large
        .cursorBatchSize(mongoCursorBatchSize)
        .build();

    data = mongoTemplate.aggregateStream(Aggregation.newAggregation(
            Aggregation.group("person_id").count().as("count")).withOptions(aggregationOptions), collectionName, YourClazz.class);

来源：https://stackoverflow.com/questions/40510855/streaming-the-result-of-an-aggregate-operation-using-spring-data-mongodb

标签

java

mongodb

spring-data-mongodb