Elasticsearch Aggregation Broken after upgrade to 1.7.3

问题

It was working before the upgrade to 1.7.3 but now it is telling me my "Data too large for [Gender]. I ran the

curl -XGET localhost:9200/_nodes/stats/indices/fielddata?fields=*

and it produced

{
    {"fielddata":{"memory_size_in_bytes":642066528,"evictions":0,
    "fields":{"Markers":{"memory_size_in_bytes":196538816},
     "RegistrationDate":{"memory_size_in_bytes":101759288},
     "Abbreviation":{"memory_size_in_bytes":185815224},
     "Gender":{"memory_size_in_bytes":52988320},
     "Birthdate":{"memory_size_in_bytes":104956384},
     "buildNum":{"memory_size_in_bytes":8496}
     }
}

Gender is only a single character so how could it grow that large?

Reading https://www.elastic.co/guide/en/elasticsearch/guide/current/_limiting_memory_usage.html seems to indicate that it should not be that big. It only has M, F, U for values. Any thoughts to the reason?

Mark

nested:  UncheckedExecutionException[org.elasticsearch.common.breaker.CircuitBreakingException: [FIELDDATA] Data too large, data for [Gender] would be larger than limit of [633785548/604.4mb]];

回答1:

You might be encountering a CircuitBreakingException. The main reason for this is that your heap is almost full and there was not enough memory available to serve the request. It doesn't necessarily mean that the Gender values take up the whole memory, just that ES was unable to load all the fielddata it needed for the Gender field with what remains of the available memory.

By default, no fielddata evictions happen (hint "evictions":0 in your output) if there's no specific limit set for the fielddata cache and by default no limit is set. So you should try setting the indices.fielddata.cache.size to a relative (e.g. 10%) or an absolute value (e.g. 3GB).

You can either set it in the elasticsearch.yml file (on each node) and restart your nodes or do it dynamically with

curl -XPUT localhost:9200/_cluster/settings -d '{
    "persistent" : {
        "indices.fielddata.cache.size" : "20%"
    }
}'

Note that setting a value that is too low will negatively impact your cluster since there will be a lot of evictions and fielddata cache will have to be rebuilt too often. So you probably need to experiment a bit until you find the right value.

来源：https://stackoverflow.com/questions/33266361/elasticsearch-aggregation-broken-after-upgrade-to-1-7-3

标签

ElasticSearch

aggregation