How to get url path using logstash on elasticsearch

问题

I have tested using my configuration of logstash

127.0.0.1 - - [02/Jun/2016:15:38:57 +0900] "GET /ad/adInfos?id=1 HTTP/1.1" 404 68

filter {
  grok {
    match => { "message" => "%{COMMONAPACHELOG}" }
  }
}

It's working as below

{
        "message" => "127.0.0.1 - - [02/Jun/2016:15:39:02 +0900] \"POST /ad/signIn?id=1 HTTP/1.1\" 200 26",
       "@version" => "1",
     "@timestamp" => "2016-06-02T06:39:02.000Z",
           "path" => "/opt/node-v4.3.1/logs/access.log",
           "host" => "0.0.0.0",
       "clientip" => "127.0.0.1",
          "ident" => "-",
           "auth" => "-",
      "timestamp" => "02/Jun/2016:15:39:02 +0900",
           "verb" => "POST",
        "request" => "/ad/signIn?id=1
    "httpversion" => "1.1",
       "response" => "200",
          "bytes" => "26"
}

But I want to get only URL path except path parameter: /ad/signIn

Because of request counting each REST API.

How can I do?

回答1:

You simply need to add a second grok after the first one that looks like this:

grok {
  match => { "request" => "%{URIPATH:path}" }
  named_captures_only => false
}

What this will do is take your request field and parse it again using the URIPATH pattern and store the result in the path field (see the last field).

{
        "message" => "127.0.0.1 - - [02/Jun/2016:15:38:57 +0900] \"GET /ad/adInfos?id=1 HTTP/1.1\" 404 68",
       "@version" => "1",
     "@timestamp" => "2016-06-03T04:54:49.631Z",
           "host" => "iMac-de-Consulthys.local",
       "clientip" => "127.0.0.1",
          "ident" => "-",
           "auth" => "-",
      "timestamp" => "02/Jun/2016:15:38:57 +0900",
           "verb" => "GET",
        "request" => "/ad/adInfos?id=1",
    "httpversion" => "1.1",
       "response" => "404",
          "bytes" => "68",
           "path" => "/ad/adInfos"
}

来源：https://stackoverflow.com/questions/37605189/how-to-get-url-path-using-logstash-on-elasticsearch

标签

logstash