Using JSON with LogStash

Question

I\'m going out of my mind here. I have an app that writes logs to a file. Each log entry is a JSON object. An example of my .json file looks like the following:

Answer 1

Try removing the json codec and adding a json filter

input {
  file {
    type => "json"
    path => "/logs/mylogs.log"
  }
}
filter{
    json{
        source => "message"
    }
}
output {
  file {
    path => "/logs/out.log"
  }
}

you do not need the json codec because you do not want decode the source JSON but you want filter the input to get the JSON data in the @message field only.

Hope this helps.

Answer 2

By default tcp put everything to message field if json codec not specified.

An workaround to _jsonparsefailure of the message field after we specify the json codec also can be rectified by doing the following:

input {
  tcp {
    port => '9563'
  }
}
filter{
  json{
    source => "message"
    target => "myroot"
  }
  json{
    source => "myroot"
  }

}
output {
    elasticsearch {
      hosts => [ "localhost:9200" ]
    }
}

It will parse message field to proper json string to field myroot and then myroot is parsed to yield the json.

We can remove the redundant field like message as

filter {
  json {
    source => "message"
    remove_field => ["message"]
  }
}

Answer 3

Try with this one:

filter {
  json {
        source => "message"
        target => "jsoncontent" # with multiple layers structure
  }
}