elasticsearch-plugin

I have the many of my logs indexed in logstash-Year-Week format. That is if i want to delete indices older than a few weeks, how can I achieve that in elasticsearch. Is there an easy, seamless way to do that? Curator would be an ideal match here. You can find the link here - https://github.com/elastic/curator A command like below should work just fine - curator --host <IP> delete indices --older-than 30 --prefix "twitter-" --time-unit days --timestring '%Y-%m-%d' You can keep in this in the CRON for removing the indices occasionally. You can find some examples and docs here - https://www

I am having trouble using sprintf to reference the event fields in the elasticsearch output plugin and I'm not sure why. Below is the event received from Filebeat and sent to Elasticsearch after filtering is complete: { "beat" => { "hostname" => "ca86fed16953", "name" => "ca86fed16953", "version" => "6.5.1" }, "@timestamp" => 2018-12-02T05:13:21.879Z, "host" => { "name" => "ca86fed16953" }, "tags" => [ [0] "beats_input_codec_plain_applied", [1] "_grokparsefailure" ], "fields" => { "env" => "DEV" }, "source" => "/usr/share/filebeat/dockerlogs/logstash_DEV.log", "@version" => "1", "prospector" =

I have been trying to add pagination in elasticsearch term aggregation. In query we can add the pagination like, { "from": 0, // to add the start to control the pagination "size": 10, "query": { } } this is pretty clear, but when I want to add pagination to aggregation, I read a lot about it, but I couldn't find anything, My code looks like this, { "from": 0, "size": 0, "aggs": { "group_by_name": { "terms": { "field": "name", "size": 20 }, "aggs": { "top_tag_hits": { "top_hits": { "size": 1 } } } } } } Is there any way to create pagination with a function or any other suggestions? Seems like

Is there any way to import a JSON file (contains 100 documents) in elasticsearch server? I want to import a big json file into es-server.. You should use Bulk API . Note that you will need to add a header line before each json document. $ cat requests { "index" : { "_index" : "test", "_type" : "type1", "_id" : "1" } } { "field1" : "value1" } $ curl -s -XPOST localhost:9200/_bulk --data-binary @requests; echo {"took":7,"items":[{"create":{"_index":"test","_type":"type1","_id":"1","_version":1,"ok":true}}]} As dadoonet already mentioned, the bulk API is probably the way to go. To transform your

I'm new to Elasticsearch and I read here https://www.elastic.co/guide/en/elasticsearch/plugins/master/mapper-attachments.html that the mapper-attachments plugin is deprecated in elasticsearch 5.0.0. I now try to index a pdf file with the new ingest-attachment plugin and upload the attachment. What I've tried so far is curl -H 'Content-Type: application/pdf' -XPOST localhost:9200/test/1 -d @/cygdrive/c/test/test.pdf but I get the following error: {"error":{"root_cause":[{"type":"mapper_parsing_exception","reason":"failed to parse"}],"type":"mapper_parsing_exception","reason":"failed to parse",