Using InMemoryConnection to test ElasticSearch

Question

I'm trying to add testing around our use of ElasticSearch (in C# using Nest 1.4.2) and want to use InMemoryConnection but I'm missing something (I assume) and having no success.

I've created this simple Nunit test case as a boiled down example of my issue

using System;
using Elasticsearch.Net.Connection;
using FluentAssertions;
using Nest;
using NUnit.Framework;

namespace NestTest
{
    public class InMemoryConnections
    {
        public class TestThing
        {
            public string Stuff { get; }

            public TestThing(string stuff)
            {
                Stuff = stuff;
            }
        }

        [Test]
        public void CanBeQueried()
        {
            var connectionSettings = new ConnectionSettings(new Uri("http://foo.test"), "default_index");

            var c = new ElasticClient(connectionSettings, new InMemoryConnection(connectionSettings));
            c.Index(new TestThing("peter rabbit"));

            var result = c.Search<TestThing>(sd => sd);

            result.ConnectionStatus.Success.Should().BeTrue();
        }
    }
}

the query is successful but doesn't find the document I just indexed...

If I update to NEST version 2.3.3 and the new syntax

        [Test]
        public void CanBeQueried()
        {

            var connectionPool = new SingleNodeConnectionPool(new Uri("http://localhost:9200"));
            var settings = new ConnectionSettings(connectionPool, new InMemoryConnection());
            settings.DefaultIndex("default");

            var c = new ElasticClient(settings);

            c.Index(new TestThing("peter rabbit"));

            var result = c.Search<TestThing>(sd => sd);

            result.CallDetails.Success.Should().BeTrue();
            result.Documents.Single().Stuff.Should().Be("peter rabbit");
        }

it fails in the same way... i.e. the query is reported as successful but returns 0 documents

Answer 1

InMemoryConnection doesn't actually send any requests or receive any responses from Elasticsearch; used in conjunction with .SetConnectionStatusHandler() on Connection settings (or .OnRequestCompleted() in NEST 2.x+), it's a convenient way to see the serialized form of requests.

When not using InMemoryConnection but still setting .SetConnectionStatusHandler() or .OnRequestCompleted(), depending on NEST version, it's a convenient way to also see the responses, when .ExposeRawResponse(true) is also set in NEST 1.x, or .DisableDirectStreaming() is set in NEST 2.x+, respectively.

An example with NEST 1.x

void Main()
{
    var settings = new ConnectionSettings(new Uri("http://localhost:9200"))
        .ExposeRawResponse(true)
        .PrettyJson()
        .SetDefaultIndex("myIndexName")
        .MapDefaultTypeNames(d => d.Add(typeof(myPoco), string.Empty))
        .SetConnectionStatusHandler(r =>
        {
            // log out the requests
            if (r.Request != null)
            {
                Console.WriteLine("{0} {1} \n{2}\n", r.RequestMethod.ToUpperInvariant(), r.RequestUrl,
                    Encoding.UTF8.GetString(r.Request));
            }
            else
            {
                Console.WriteLine("{0} {1}\n", r.RequestMethod.ToUpperInvariant(), r.RequestUrl);
            }

            Console.WriteLine();

            if (r.ResponseRaw != null)
            {
                Console.WriteLine("Status: {0}\n{1}\n\n{2}\n", r.HttpStatusCode, Encoding.UTF8.GetString(r.ResponseRaw), new String('-', 30));
            }
            else
            {
                Console.WriteLine("Status: {0}\n\n{1}\n", r.HttpStatusCode, new String('-', 30));
            }
        });

    var client = new ElasticClient(settings, new InMemoryConnection());

    int skipCount = 0;
    int takeSize = 100;

    var searchResults = client.Search<myPoco>(x => x
        .Query(q => q
        .Bool(b => b
        .Must(m =>
            m.Match(mt1 => mt1.OnField(f1 => f1.status).Query("New")))))
        .Skip(skipCount)
        .Take(takeSize)
    );
}

public class myPoco
{
    public string status { get; set;}
}

yields

POST http://localhost:9200/myIndexName/_search?pretty=true 
{
  "from": 0,
  "size": 100,
  "query": {
    "bool": {
      "must": [
        {
          "match": {
            "status": {
              "query": "New"
            }
          }
        }
      ]
    }
  }
}


Status: 0
{ "USING NEST IN MEMORY CONNECTION"  : null }

------------------------------

And for NEST 2.x

void Main()
{
    var pool = new SingleNodeConnectionPool(new Uri("http://localhost:9200"));
    var defaultIndex = "default-index";
    var connectionSettings = new ConnectionSettings(pool, new InMemoryConnection())
            .DefaultIndex(defaultIndex)
            .PrettyJson()
            .DisableDirectStreaming()
            .OnRequestCompleted(response =>
                {
                    // log out the request
                    if (response.RequestBodyInBytes != null)
                    {
                        Console.WriteLine(
                            $"{response.HttpMethod} {response.Uri} \n" +
                            $"{Encoding.UTF8.GetString(response.RequestBodyInBytes)}");
                    }
                    else
                    {
                        Console.WriteLine($"{response.HttpMethod} {response.Uri}");
                    }

                    Console.WriteLine();

                    // log out the response
                    if (response.ResponseBodyInBytes != null)
                    {
                        Console.WriteLine($"Status: {response.HttpStatusCode}\n" +
                                 $"{Encoding.UTF8.GetString(response.ResponseBodyInBytes)}\n" +
                                 $"{new string('-', 30)}\n");
                    }
                    else
                    {
                        Console.WriteLine($"Status: {response.HttpStatusCode}\n" +
                                 $"{new string('-', 30)}\n");
                    }
                });

    var client = new ElasticClient(connectionSettings);

    int skipCount = 0;
    int takeSize = 100;

    var searchResults = client.Search<myPoco>(x => x
        .Query(q => q
        .Bool(b => b
        .Must(m =>
            m.Match(mt1 => mt1.Field(f1 => f1.status).Query("New")))))
        .Skip(skipCount)
        .Take(takeSize)
    );
}

You can of course mock/stub responses from the client using your favourite mocking framework and depending on the client interface, IElasticClient, if that's a route you want to take, although asserting the serialized form of a request matches your expectations in the SUT may be sufficient.