RavenDb : Force indexes to wait until not stale whilst unit testing

Question

When unit testing with RavenDb, it is often the case that newly added data is retrieved or otherwise processed. This can lead to 'stale index' exceptions e.g.

Bulk operation cancelled because the index is stale and allowStale is false

According to a number of answers

• How should stale indexes be handled during testing?
• WaitForNonStaleResults per DocumentStore
• RavenDb : Update a Denormalized Reference property value

The way to force the database (the IDocumentStore instance) to wait until its indexes are not stale before processing a query or batch operation is to use DefaultQueryingConsistency = ConsistencyOptions.QueryYourWrites during the IDocumentStore initialisation, like this:

public class InMemoryRavenSessionProvider : IRavenSessionProvider
{
    private static IDocumentStore documentStore;

    public static IDocumentStore DocumentStore
    {
        get { return (documentStore ?? (documentStore = CreateDocumentStore())); }
    }

    private static IDocumentStore CreateDocumentStore()
    {
        var store = new EmbeddableDocumentStore
        {
            RunInMemory = true,
            Conventions = new DocumentConvention
            {
                DefaultQueryingConsistency = ConsistencyOptions.QueryYourWrites,
                IdentityPartsSeparator = "-"
            }
        };
        store.Initialize();
        IndexCreation.CreateIndexes(typeof (RavenIndexes).Assembly, store);
        return store;
    }

    public IDocumentSession GetSession()
    {
        return DocumentStore.OpenSession();
    }
}

Unfortunately, the code above does not work. I am still receiving exceptions regarding stale indexes. These can be resolved by putting in dummy queries that include .Customize(x => x.WaitForNonStaleResultsAsOfLastWrite()).

This is fine, as long as these can be contained in the Unit Test, but what if they can't? I am finding that these WaitForNonStaleResults* calls are creeping into production code just so I can get unit-tests to pass.

So, is there a sure fire way, using the latest version of RavenDb, to force the indexes to freshen before allowing commands to be processed - for the purposes of unit testing only?

Edit 1

Here is a solution based on the answer give below that forces a wait until the index is not stale. I have written it as an extension method for the sake of unit-testing convenience;

public static class IDocumentSessionExt
{
    public static void ClearStaleIndexes(this IDocumentSession db)
    {
        while (db.Advanced.DatabaseCommands.GetStatistics().StaleIndexes.Length != 0)
        {
            Thread.Sleep(10);
        }
    }
}

And here is a Unit Test that was using the verbose WaitForNonStaleResultsAsOfLastWrite technique but now uses the neater extension method.

[Fact]
public void Should_return_list_of_Relationships_for_given_mentor()
{
    using (var db = Fake.Db())
    {
        var mentorId = Fake.Mentor(db).Id;
        Fake.Relationship(db, mentorId, Fake.Mentee(db).Id);
        Fake.Relationship(db, mentorId, Fake.Mentee(db).Id);
        Fake.Relationship(db, Fake.Mentor(db).Id, Fake.Mentee(db).Id);

        //db.Query<Relationship>()
        //  .Customize(x => x.WaitForNonStaleResultsAsOfLastWrite())
        //  .Count()
        //  .ShouldBe(3);

        db.ClearStaleIndexes();
        db.Query<Relationship>().Count().ShouldBe(3);
        MentorService.GetRelationships(db, mentorId).Count.ShouldBe(2);
    }
}

Answer 1

If you have a Map/Reduce index, DefaultQueryingConsistency = ConsistencyOptions.QueryYourWrites won't work. You need to use an alternative method.

In your units tests, call code like this, straight after you've inserted any data, this will force the all indexes to update before you do anything else:

while (documentStore.DatabaseCommands.GetStatistics().StaleIndexes.Length != 0)
{
    Thread.Sleep(10);
}

Update You can of course put it in an extension method if you want to:

public static class IDocumentSessionExt
{
    public static void ClearStaleIndexes(this IDocumentSession db)
    {
        while (db.Advanced.DatabaseCommands.GetStatistics().StaleIndexes.Length != 0)
        {
            Thread.Sleep(10);
        }
    }
}

Then you can say:

db.ClearStaleIndexes();

Answer 2

You can actually add a query listener on the DocumentStore to wait for nonstale results. This can be used just for unit tests as it is on the document store and not each operation.

// Initialise the Store.
var documentStore = new EmbeddableDocumentStore
                {
                    RunInMemory = true
                };
documentStore.Initialize();

// Force queries to wait for indexes to catch up. Unit Testing only :P
documentStore.RegisterListener(new NoStaleQueriesListener());

....


#region Nested type: NoStaleQueriesListener

public class NoStaleQueriesListener : IDocumentQueryListener
{
    #region Implementation of IDocumentQueryListener

    public void BeforeQueryExecuted(IDocumentQueryCustomization queryCustomization)
    {
        queryCustomization.WaitForNonStaleResults();
    }

    #endregion
}

#endregion

(Shamelessly stolen from RavenDB how to flush?)

Answer 3

Be aware that StaleIndexes also include abondoned and disabled indices - which will never get up to date.

So to avoid waiting indefinetely use this property instead:

 var staleIndices = store.DatabaseCommands.GetStatistics().CountOfStaleIndexesExcludingDisabledAndAbandoned;