Unable to get Log4J SocketAppender Working

Question

My Java project uses Log4J2. Currently, it is successfully writing logs to a file. Now, I'm trying to push the logs to LogStash via a Socket Appender. Unfortunately, I am not having any success with these efforts. At this time, I'm looking at two pieces: my log4j2.xml file and my logstash config file. I've provided both here in hopes that someone can help me identify my problem.

log4j2.xml

<Configuration status="WARN" monitorInterval="30">
  <Appenders>
    <Socket name="A1" host="0.0.0.0" port="4560">
      <SerializedLayout/>
    </Socket>

    <RollingRandomAccessFile name="RollingFile" fileName="/logs/server.log"
                             filePattern="/logs/$${date:yyyy-MM}/server-%d{yyyy-MM-dd-HH}-%i.log.gz">
      <JSONLayout complete="false"></JSONLayout>
      <Policies>
        <TimeBasedTriggeringPolicy interval="6" modulate="true"/>
        <SizeBasedTriggeringPolicy size="100 MB"/>
      </Policies>
    </RollingRandomAccessFile>
    <Async name="AsyncFile">
      <AppenderRef ref="RollingFile"/>
    </Async>
  </Appenders>
  <Loggers>
    <Logger name="com.company" level="trace" additivity="false">
      <AppenderRef ref="AsyncFile"/>
    </Logger>
    <Root level="trace">
      <AppenderRef ref="AsyncFile"/>
    </Root>
  </Loggers>
</Configuration>

That was my log4j2 configuration. Here is my logstash configuration:

logstash.conf

input {
  log4j {
    mode => "server"
    type => "log4j"
    host => "0.0.0.0"
    port => "4560"
  }
}
output {
  stdout {
    codec => "json"
  }
}

I start logstash from the command-line using logstash-1.4.0/bin/logstash --config /etc/logstash/logstash.conf --debug. I do not see any errors. At the same time, I do not see any logs written to the console window. I know that logs should appear because they are appearing in my rolling server.log file that was configured as the second appender in log4j.

What am I doing wrong? I've been tinkering with this for 3 days.

Answer 1

The log4j input in Logstash is not compatible with Log4j2. Log4j2 uses a new format to serialize the log data.

There is however a plugin which enables parsing of Log4j2 sockets: https://github.com/jurmous/logstash-log4j2

In logstash 1.5+ it can be installed from: https://rubygems.org/gems/logstash-input-log4j2

Answer 2

I think you should replace 0.0.0.0 with your IP on the following line:

<Socket name="A1" host="0.0.0.0" port="4560">

And add the following lines in elements <Root ...> and <Logger ...> (like you did for AsyncFile appender):

<AppenderRef ref="A1" />
<AppenderRef ref="RollingFile" />