Parse Apache2 Error logs with Grok for Logstash

Question

Im trying to parse my apache2 error log and im having a bit of trouble.. It doesnt seem to be matching the filter. Im pretty sure the timestamp piece is wrong, but im not sure, and i cant really find any documentation to figure it out. Also, is there a way to get what is in fields.errmsg to me @message?

Log

[Wed Jun 26 22:13:22 2013] [error] [client 10.10.10.100] PHP Fatal error:  Uncaught exception '\Foo\Bar'

Shipper Config

input {
        file {
                'path' => '/var/log/apache2/*-error.log'
                'type' => 'apache-error'
        }

}

filter {
        grok {
                type => "apache-error"
                pattern => "\[%{HTTPDATE:timestamp}\] \[%{WORD:class}\] \[%{WORD:originator} %{IP:clientip}\] %{GREEDYDATA:errmsg}"
        }
}


output {
        stdout {}
        redis {
                'data_type' => 'list'
                'host' => 'logstash.server.net'
                'key' => 'logstash'
        }
}

Answer 1

Ahoy!

I know I'm a little late to the party, but here it goes!

I created a /etc/logstash/patterns.d/ directory on the system and thew a file named apache-error in it containing:

APACHE_ERROR_TIME %{DAY} %{MONTH} %{MONTHDAY} %{TIME} %{YEAR}
APACHE_ERROR_LOG \[%{APACHE_ERROR_TIME:timestamp}\] \[%{LOGLEVEL:loglevel}\] (?:\[client %{IPORHOST:clientip}\] ){0,1}%{GREEDYDATA:errormsg}

/etc/logstash/patterns.d/ will be referenced in the logstash configuration as follows:

grok {
  patterns_dir => [ "/etc/logstash/patterns.d" ]
  match => [ "message", "%{APACHE_ERROR_LOG}" ]
}

You can test it out at Grok Debugger, like Adam mentioned in his comment. Seems to work just fine with the sample log entry you send. Has been working pretty solidly me with one of my clients.

The pattern above puts the final message in errormsg field. So I just delete the message field.

This is what I am currently using in my logstash configuration:

filter {
  if [type] == "apache_error_log" {

    grok {
      patterns_dir => [ "/etc/logstash/patterns.d" ]
      match => [ "message", "%{APACHE_ERROR_LOG}" ]
    }

    if !("_grokparsefailure" in [tags]) {

      mutate {
        remove_field => [ "message" ]
        add_field =>  ["timestamp_submitted", "%{@timestamp}"]
      }

      date {
        # Try to pull the timestamp from the 'timestamp' field (parsed above with
        # grok). The apache time format looks like: "18/Aug/2011:05:44:34 -0700"
        #                        Sat Feb 08 06:31:09 2014
        match => [ "timestamp", "EEE MMM dd HH:mm:ss yyyy" ]
        remove_field => [ "timestamp" ]
      }

      geoip {
        source => "clientip"
      }
    }
  }
}

Note that I am using a type of apache_error_log instead of apache-error.

Give it s shot. I would love to know if this works for you and others!

Cheers!

Answer 2

Our error logs look a little different:

[Tue Dec 08 12:30:35.997887 2015] [ssl:info] [pid 1178:tid 1072] (70014)End of file found: [client 10.129.24.77:49987] AH01991: SSL input filter read failed.

But I found a predefined pattern that works perfectly:

HTTPD24_ERRORLOG

See this in the Logstash source

Answer 3

Didn't work for me:

EEE MMM dd HH:mm:ss yyyy

Worked for me (Apache 2.4):

EEE MMM dd HH:mm:ss.SSSSSS yyyy

Answer 4

Logstash has a built-in apache log parser.

Here is an example...

grok {
   type    => 'company'
   pattern => ["%{COMBINEDAPACHELOG}"]
   add_tag => "apache"
}

As a reference, you can check Logstash's docs