Logstash - Send output from log files to elk

问题

I have an index in elastic search that has a field named locationCoordinates. It's being sent to ElasticSearch from logstash.

The data in this field looks like this...

-38.122, 145.025

When this field appears in ElasticSearch it is not coming up as a geo point.

I know if I do this below it works.

{
  "mappings": {
    "logs": {
      "properties": {
        "http_request.locationCoordinates": {
          "type": "geo_point"
        }
      }
    }
  }
}

But what I would like to know is how can i change my logstash.conf file so that it does this at startup.

At the moment my logstash.conf looks a bit like this...

input {

    # Default GELF input
    gelf {
        port => 12201
        type => gelf
    }

    # Default TCP input
    tcp {
        port => 5000
        type => syslog
    }

    # Default UDP input
    udp {
        port => 5001
        type => prod
        codec => json
    }
    file {
       path =>  [ "/tmp/app-logs/*.log" ]
       codec =>   json {
          charset => "UTF-8"
       }
       start_position => "beginning"
       sincedb_path => "/dev/null"
   }
}

filter {
   json{
      source => "message"
   }
}

output {
    elasticsearch {
        hosts => "elasticsearch:9200"
    }
}

And I end up with this in Kibana (without the little Geo sign).

回答1:

You simply need to modify your elasticsearch output to configure an index template in which you can add your additional mapping.

output {
    elasticsearch {
        hosts => "elasticsearch:9200"
        template_overwrite => true
        template => "/path/to/template.json"
    }
}

And then in the file at /path/to/template.json you can add your additional geo_point mapping

{
  "template": "logstash-*",
  "mappings": {
    "logs": {
      "properties": {
        "http_request.locationCoordinates": {
          "type": "geo_point"
        }
      }
    }
  }
}

If you want to keep the official logstash template, you can download it and add your specific geo_point mapping to it.

来源：https://stackoverflow.com/questions/41216038/logstash-send-output-from-log-files-to-elk

标签

ElasticSearch

logstash