ElasticSearch dump full index to a csv file

Question

Is it possible to easily export whole index(all stored fields) of an elasticsearch cluster in a .csv file(possibly out of the box)?

The alternative I can think is query the whole index and then convert the result... but I really don't like the idea of parsing a monstrous json since it contains some millions of documents!

Are there any other ways or ideas to achieve the export?

Answer 1

Disclaimer: I'm the author of estab.

---

estab exports elasticsearch fields as tab separated values. If you do not have too many fields, it's easy to explicitly export them all. Internally estab uses the scan and scroll API.

Example:

Assume your index contains documents like:

{
    "name": "Kiwi",
    "genus": "Apteryx",
    "populations": [
        {"location": "North Island", "size": 2500},
        {"location": "Little Barrier Island", "size": 2000}
    ]
}

Then you can export a whole index via:

$ estab -f 'name genus populations.location populations.size'
Kiwi    Apteryx North Island|Little Barrier Island  2500|2000

Note that nested fields are not easily mapped into a tabular form. estab by default separates multiple values by | and you can refer to nested fields via usual dot notation.

Answer 2

You could use the Scan and Scroll API. Then you wouldn't have a monstrous JSON file to parse--you could do it in much smaller batches.