ElasticSearch and Nest: Why amd I missing the id field on a query?

Question

I created a simple object that represents a Meeting that has elements such as time, location, name, topic, etc and indexed it in ElasticSearch via Nest. It has an Id field that I leave blank so that ES can generate them.

Later on I retrieved all the documents that are missing GEO coordinates so that I may update them. All my returned elements still have null for the id field and when I update them back to ES it creates new documents for them.

What am I missing here that makes all my id's null?

Thank you

Here is the Meeting class (the id prop is redundant but I tried it anyway)

[ElasticType(IdProperty = "Id")]
    public class Meeting
    {
        public string Id { get; set; }
        public string Code { get; set; }
        public string Day { get; set; }
        public string Town { get; set; }
        public string Name { get; set; }
        public string Location { get; set; }
        public string OriginalTime { get; set; }
        public string OriginalTimeCleaned { get; set; }
        public string Handicap { get; set; }
        public string FormattedAddress { get; set; }
        public Coordinates Coordinates { get; set; }
        public List<MeetingTime> Times = new List<MeetingTime>();
        public bool IsProcessed { get; set; }    
    }

Here is how I retrieve meetings

 public static List<Meeting> GetAddressesWithMissingCoordinates()
        {

            var result = Client.Search<Meeting>(s => s
                .Index("meetings")
                .AllTypes()
                .Query(p => p.Filtered(f => f.Filter(x => x.Missing(c => c.Coordinates)))));


            return result.Documents.ToList();
        }

Here is my update statement, Id is null

 public static void UpdateMeetingCoordinates(Meeting meeting, Coordinates coordinates)
        {
            meeting.Coordinates = coordinates;

            var response = Client.Index(meeting, u => u
                .Index("meetings")
                .Type("meeting")
                //.Id(meeting.Id.ToString())
                .Refresh()
                );

            Console.WriteLine(response);
        }

I've tried partial updates as well with no luck.

Answer 1

There is a way to get the internal id, as described in this issue requesting this very feature.

Rather than using response.Documents, do this instead:

var results = response.Hits.Select(hit =>
    {
        var result = hit.Source;
        result.Id = hit.Id;
        return result;
    });

Answer 2

Elasticsearch sets an "_id" meta-data parameter (for which it chooses a value if you don't specify one), but it doesn't set that value in your document source.

To illustrate, if I create a trivial index:

PUT /test_index

then give it a couple of documents, without specifying "_id":

POST /test_index/doc/_bulk
{"index":{}}
{"id":null,"name":"doc1"}
{"index":{}}
{"id":null,"name":"doc2"}

and then search:

POST /test_index/_search

this is what I get back:

{
   "took": 2,
   "timed_out": false,
   "_shards": {
      "total": 5,
      "successful": 5,
      "failed": 0
   },
   "hits": {
      "total": 2,
      "max_score": 1,
      "hits": [
         {
            "_index": "test_index",
            "_type": "doc",
            "_id": "AVEmuVlmj_RE0PsHCpza",
            "_score": 1,
            "_source": {
               "id": null,
               "name": "doc2"
            }
         },
         {
            "_index": "test_index",
            "_type": "doc",
            "_id": "AVEmuVlmj_RE0PsHCpzZ",
            "_score": 1,
            "_source": {
               "id": null,
               "name": "doc1"
            }
         }
      ]
   }
}

Notice that the "_id" meta-data parameter was set for both documents, but the "id" field I passed is unchanged. That's because, as far as Elasticsearch is concerned, "id" is just another document field.

(here is the code I used: http://sense.qbox.io/gist/777dafae88311c4105453482050c64d69ccd09db)