Filtering specific lines

问题

I am currently trying to filter specific lines from my log file. My lines in the log file are of the following pattern.

[8/05/13 14:24:55.468] RuntimeErrorI E LaError
[8/05/13 14:24:55.468] AbcdEfg W SomeWarning

where the first is the date, time, application name and the Log level( WARNING, ERROR, TRACE etc) followed by the error message or warning message or any other messages.

So what I am trying to get is the log level errors only and not other log levels. I have the following which I am playing around with but I am not getting console output at all. I think I am making a mistake somewhere in my grep for checking if its E(Error)

input {

  file {
  type => "database"
  path => "/home/nakampe/Desktop/file.log"
  }

}

filter {

 grok {

    pattern =>  "[%{MONTHDAY:date} / %{MONTH:month} / %{YEAR:year}  %{TIME:time}] %WORD:application} %{WORD:levelType}  %{WORD:message}"

  }


#Here I want to only consider log levels of E (ERROR) and not others
  grep{

  match => ["levelType", "E"]

  }

}

output {

   elasticsearch { 

   embedded => true 
   }

stdout { 
  message => "%{@message}"
  }     
}

回答1:

 grok { 

pattern =>  "%{MONTHDAY:date}/%{MONTHNUM:month}/%{YEAR:year} %{TIME:time} %{WORD:sast}]      

    %{INT:threadId} %{WORD:applicationName}%{SPACE:space}%{WORD:logLevel} %     {WORD:errorMessage}"

}

My mistake was with the pattern, after debugging using http://grokdebug.herokuapp.com/ all was ok. A very useful tool.

回答2:

I think you need to chain both filters otherwise they will be applied independently to the original line. You can do this by adding a tag in the grok filter and react to this in the grep filter:

grok {
  pattern => ...
  add_tag => [ "groked_input" ]
}

and

grep {
  match => ...
  tags => [ "groked_input"]
}

Hope that helps! :)

回答3:

under the grok section

if [levelType] != "ERROR" {
        drop { }
}

来源：https://stackoverflow.com/questions/18208412/filtering-specific-lines

标签

logstash