Remove Field from event by pattern

Question

So I\'m using a standard ELK stack to analyse Apache access logs, which is working well, but I\'m looking to break out URL parameters as fields, using the KV filter, in orde

Answer 1

If the set of fields that you are interested in is known and well-defined you could set target for the kv filter, move the interesting fields to the top level of the message with a mutate filter and delete the field with the nested key/value pairs. I think this is pretty much what you suggested at the end.

Alternatively you could use a ruby filter:

filter {
  ruby {
    code => "
      event.to_hash.keys.each { |k|
        if k.start_with?('rand')
          event.remove(k)
        end
      }
    "
  }
}