Filter a static RavenDB map/reduce index

Question

Scenario/Context

• Raven 2.0 on RavenHQ
• Web app, so async is preferred

My application is a survey application. Each Survey has an array of Questions; and conversely, each Submission (an individual's response to a survey) has an array of Answers.

I have a static index that aggregates all answers so that I can display a chart based on the responses (e.g. for each question on each survey, how many people selected each option). These data are used to render, for example, a pie chart. This aggregation index (discussed in this question) basically gives an object per question per survey, with sums for each option.

Problem

I would like to filter these aggregated values. Some of them are trivial because they are fields in the result (e.g. filter by SurveyId or QuestionId). However, I'd also like to filter by Submission date (from metadata), or by LocationId, which are fields in the individual Submissions, but obviously not in the aggregation results.

In other words, I need to be able to ask Raven about the results to question for a specific LocationId, or during this month.

Classes

Here is what a single Submission basically looks like:

{
  "SurveyId": 1,
  "LocationId": 1,
  "Answers": [
    {
      "QuestionId": 1,
      "Values": [2,8,32],
      "Comment": null
    },
    {
      "QuestionId": 2,
      "Values": [4],
      "Comment": "Lorem ipsum"
    },
    ...more answers...
  ]
}

Currently, here's the Aggregation Result:

public class Result
{
    public int SurveyId { get; set; } // 1
    public int QuestionId { get; set; } // 1
    public int NumResponses { get; set; } // 576
    public int NumComments { get; set; } // 265
    public IList<KeyValuePair<int,int>> Values { get; set; } // [{Key:1, Value:264}, {Key:2, Value:163}, Key:4, Value:391}, ...]
}

Here's the Aggregation index:

Map = submissions => 
    from submission in submissions
    from answer in submission.Answers
    select new
    {
        submission.SurveyId,
        answer.QuestionId,
        NumResponses = 1,
        NumComments = answer.Comment == null ? 0 : 1,
        Value = answer.Value.Select(x => new KeyValuePair<int, int>(x, 1))
    };

Reduce = results => 
    from result in results
    group result by new { result.SurveyId, result.QuestionId }
        into g
        select new Result
        {
            SurveyId = g.Key.SurveyId,
            QuestionId = g.Key.QuestionId,
            NumResponses = g.Sum(x => x.NumResponses),
            NumComments = g.Sum(x => x.NumComments),
            Value = g.SelectMany(x => x.Value)
                        .GroupBy(x => x.Key)
                        .Select(x => new KeyValuePair<int, int>(x.Key, x.Sum(y => y.Value)))
        };

I'm inclined, conceptually, to "pass in" these filters to the query, but from what I've read, this will not work because the index values are indexed (stored) asynchronously without the individual Submission dates or LocationIds.

Does this mean that I'll need to create an index of all answers, and then have the aggregation index query this new AllAnswers index, or something? I've done a little bit of searching for having one index query another, with no luck. Or is this what Fields are used for??

Any guidance is appreciated!

Answer 1

The index you have currently aggregates all of the data together, by the SurveyId and QuestionId. If you want it broken out by date or location, those are new indexes. You would simply add the fields you wanted to the map, include them in the grouping key, and pass them through in the result. Then you can easily query by those keys.

When you have different grouping keys, you can't consolidate that into a single index. You have to have multiple indexes. For example, I might call the index you have above Submission_TotalsBySurveyAndQuestion, and another index might be Submission_TotalsBySurveyAndQuestionPerLocation.

Think of it this way - right now, you can include a Where or OrderBy in your query that goes against the SurveyId or QuestionId - because those are grouping keys in your index. If you want to filter or sort by LocationId, that has to be included.

One word of caution, you said:

I'd also like to filter by Submission date (from metadata)

The only date RavenDB gives you in the metadata (by default) is the Last-Modified date. Any edit to the document will update this. So if the submission date is important to you, then you should probably keep it in your own property.