Filter specific Message with logstash before sending to ElasticSearch

问题

I had like to know if it is possible to send only specific log messages to elasticsearch via logstash? E.G let's say I have these messages in my log file:

2015-08-14 12:21:03 [31946] PASS  10.249.10.70  http://google.com
2015-08-14 12:25:00 [2492]  domainlist \"/etc/ufdbguard/blacklists\
2015-08-14 12:21:03 [31946] PASS 10.249.10.41 http://yahoo.com

I had like to skip the second line when logstash/log forwarder process this log, is it possible to instruct it to skip any log message with the keyword 'domainlist'? Or allow only log messages with the keyword 'PASS'?

回答1:

Yes, you can achieve that by using the drop filter.

Depending on how you grok your log line and which field names you have, you can decide to drop an event if it matches some criteria. For instance, below you can see a conditional after the grok filter, which checks whether myfield contains something different than the value PASS in which case it will drop the event.

filter {
  grok {
      ...your parsing regexp here...
  }

  if [myfield] != "PASS" {
    drop { }
  }
}

回答2:

Although Val's answer is correct, another way you can do it is by filtering with a conditional:

output {
  if "PASS" in [message] {
    elasticsearch {
      ...
    }
  }
}

来源：https://stackoverflow.com/questions/32370265/filter-specific-message-with-logstash-before-sending-to-elasticsearch