logstash-grok

问题 So, I have a hostname that is being set by filebeat (and I've written a regex that should grab it), but the following isn't adding fields the way that I think it should.. grok{ patterns_dir => "/config/patterns" match =>{ "beat.hostname" => ["%{INSTALLATION}-%{DOMAIN}-%{SERVICE}"] } add_field => { "[installation]" => "%{INSTALLATION}"} add_field => { "[domain]" => "%{DOMAIN}"} add_field => { "[service]" => "%{SERVICE}"} } I can't seem to access beat.hostname, hostname, host or anything like

问题 I have set up ELK on my laptop and I am having trouble with the timestamp field. My input file looks like this ... (one line so far) Chckpoint 502 10.189.7.138 Allow 18 Mar 2015 15:00:01 My code looks like this .. input { file { path => "/usr/local/bin/firewall_log" } } filter { grok { match => {"message", "%{WORD:type} %{NUMBER:nums} %{IP:sourceip} %{WORD:Action}"} add_tag => "checkpoint" } date { match => {"DATETIME" => "%{dd mmm yyyy hh:mm:ss}"} target => "@timestamp" } } output {

问题 My original data. { message: { data: "["1,2","3,4","5,6"]" } } Now I want to convert value of data field to an array. So it should become: { message: { data: ["1,2", "3,4", "5,6"] } } By using mutate { gsub => ["data", "[\[\]]", ""] } I got rid of square brackets. After this, I tried splitting based on commas. But that won't work. Since my data has commas as well. I tried writing a dissect block but that is not useful. So how should I go ahead with this? 回答1: Have you tried the json filter?

问题 I'm using logstash, filebeat and grok to send data from logs to my elastisearch instance. This is the grok configuration in the pipe filter { grok { match => { "message" => "%{SYSLOGTIMESTAMP:messageDate} %{GREEDYDATA:messagge}" } } } This works fine, the issue is that messageDate is in this format Jan 15 11:18:25 and it doesn't have a year entry. Now, i actually know the year these files were created in and i was wondering if it is possible to add the value to the field during the process,

问题 I'm trying to use a lookbehind and a lookahead in a Grok custom pattern and getting pattern match errors in the Grok debugger that I cannot resolve. This is for archiving system logs. I am currently trying to parse the postgrey application. Given data such as: 2019-04-09T11:41:31-05:00 67.157.192.7 postgrey: action=pass, reason=triplet found, delay=388, client_name=unknown, client_address=103.255.78.9, sender=members@domain.com, recipient=person@domain.com I'm trying to use the following to

问题 I am using Logstash to output JSON message to an API. On Simple Log lines, my grok pattern and configurations are working absolutely fine, But I am unable to get the values dynamically out during exceptions and stacktraces. Log File : TID: [-1234] [] [2016-06-07 12:52:59,862] INFO {org.apache.synapse.core.axis2.ProxyService} - Successfully created the Axis2 service for Proxy service : TestServiceHttp {org.apache.synapse.core.axis2.ProxyService} TID: [-1234] [] [2016-06-07 12:59:04,893] INFO

问题 [2018-02-12 09:15:43] development.WARNING: home page [2018-02-12 09:15:43] development.INFO: home page [2018-02-12 10:22:50] development.WARNING: home page [2018-02-12 10:22:50] development.INFO: home page [2018-02-12 10:22:50] development.ERROR: Call to undefined function vie() {"exception":"[object](Symfony\\Component\\Debug\\Exception\\FatalThrowableError(code: 0): Call to undefined function vie() at /var/www/html/routes/web.php:16 [stacktrace] #0 /var/www/html/vendor/laravel/framework/src

问题 Supposed I have this log file: Jan 1 22:54:17 drop %LOGSOURCE% >eth1 rule: 7; rule_uid: {C1336766-9489-4049-9817-50584D83A245}; src: 70.77.116.190; dst: %DSTIP%; proto: tcp; product: VPN-1 & FireWall-1; service: 445; s_port: 2612; Jan 1 22:54:22 drop %LOGSOURCE% >eth1 rule: 7; rule_uid: {C1336766-9489-4049-9817-50584D83A245}; src: 61.164.41.144; dst: %DSTIP%; proto: udp; product: VPN-1 & FireWall-1; service: 5060; s_port: 5069; Jan 1 22:54:23 drop %LOGSOURCE% >eth1 rule: 7; rule_uid:

问题 Input log looks like this, which contains data which are "|" sperated. The data contains id | type | request | response 110000|read|<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" xmlns:web="http://webservices.lookup.sdp.bharti.ibm.com"> <soapenv:Header/> <soapenv:Bod<web:getLookUpServiceDetails> <getLookUpService> <serviceRequester>iOBD</serviceRequester> <lineOfBusiness>mobility</lineOfBusiness> <lookupAttribute> <searchAttrValue>911425152231426</searchAttrValue>

问题 This is my log: 68.192.186.96 - - [18/May/2015:12:54:42 +0000] GET http://test.com/sectionId/592/apiVersion/2/type/json HTTP/1.1 200 575 \"-\" \"Mozilla/5.0 (Windows NT 6.3; WOW64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/42.0.2311.152 Safari/537.36\" \"icon_seq=0; PHPSESSID=frmnhfrrc25ullikbv71thc283\" this is my pattern %{IPORHOST:remoteip} \- \- \[%{HTTPDATE:timestamp}\] %{WORD:verb} %{NOTSPACE:request} HTTP/%{NUMBER:httpversion} %{NUMBER:status} %{NUMBER:requestNum} \"\-\" %