Filter log file entries based on date range

Question

My server is having unusually high CPU usage, and I can see Apache is using way too much memory. I have a feeling, I\'m being DOS\'d by a single IP - maybe you can help me f

Answer 1

As this is a common perl task

And because this is not exactly same than extract last 10 minutes from logfile where it's about a bunch of time upto the end of logfile.

And because I've needed them, I (quickly) wrote this:

#!/usr/bin/perl -ws
# This script parse logfiles for a specific period of time

sub usage {
    printf "Usage: %s -s= [-e=] \n";
    die $_[0] if $_[0];
    exit 0;
}

use Date::Parse;

usage "No start time submited" unless $s;
my $startim=str2time($s) or die;

my $endtim=str2time($e) if $e;
$endtim=time() unless $e;

usage "Logfile not submited" unless $ARGV[0];
open my $in, "<" . $ARGV[0] or usage "Can't open '$ARGV[0]' for reading";
$_=<$in>;
exit unless $_; # empty file
# Determining regular expression, depending on log format
my $logre=qr{^(\S{3}\s+\d{1,2}\s+(\d{2}:){2}\d+)};
$logre=qr{^[^\[]*\[(\d+/\S+/(\d+:){3}\d+\s\+\d+)\]} unless /$logre/;

while (<$in>) {
    /$logre/ && do {
        my $ltim=str2time($1);
        print if $endtim >= $ltim && $ltim >= $startim;
    };
};

This could be used like:

./timelapsinlog.pl -s=09:18 -e=09:24 /path/to/logfile

for printing logs between 09h18 and 09h24.

./timelapsinlog.pl -s='2017/01/23 09:18:12' /path/to/logfile

for printing from january 23th, 9h18'12" upto now.

In order to reduce perl code, I've used -s switch to permit auto-assignement of variables from commandline: -s=09:18 will populate a variable $s wich will contain 09:18. Care to not miss the equal sign = and no spaces!

Nota: This hold two diffent kind of regex for two different log standard. If you require different date/time format parsing, either post your own regex or post a sample of formatted date from your logfile

^(\S{3}\s+\d{1,2}\s+(\d{2}:){2}\d+)         # ^Jan  1 01:23:45
^[^\[]*\[(\d+/\S+/(\d+:){3}\d+\s\+\d+)\]    # ^... [01/Jan/2017:01:23:45 +0000]