How to use Stackdriver Structured Logging in App Engine Flex Java environment

问题

Google App Engine flexible environment automatically pipes stdout and stderr to Stackdriver (Google Cloud Logging). But this only supports plain text log message without any metadata (not even logging levels).

I found Logback appender for google cloud logging.

https://cloud.google.com/logging/docs/setup/java

But this does not seem to support structured logging yet. And also, it makes GRPC calls for every log entry under the hood. So, I wonder how scalable it is (especially compare to current app engine structure which has separate Fluentd agent handles logs).

Is there any out of box or simple solution to send structured log from App Engine to Stackdriver?

回答1:

It's quite possible to do structured logging in StackDriver logging. In fact, each individual log entry uses the LogEntry structure. As you can see it includes severity and metadata.

According to the App Engine documentation, the following logs are provided out of the box:

Request logs record requests sent to all App Engine apps. The request log is provided by default and you cannot opt out of receiving it.

App logs record activity by software within the App Engine app. The log is provided by default and you cannot opt out of receiving it.

Runtime logs are provided from the flexible environment using a preinstalled Logging agent.

回答2:

You could use the Logback appender + Implement LoggingEnhacer

public class LogEnhancer implements LoggingEnhancer {


  @Override
  public void enhanceLogEntry(LogEntry.Builder logEntry) {
    // add Labels
    logEntry.addLabel("project", "conversational-services");

    // Transform textPayload to JSONPayload
    ObjectMapper mapper = new ObjectMapper();
    Builder structBuilder = Struct.newBuilder();
    String textPayload = logEntry.build().getPayload().getData().toString();
    try {
      // Validate JSON Payload
      mapper.readTree(textPayload);
      JsonFormat.parser().merge(textPayload, structBuilder);
      logEntry.setPayload(JsonPayload.of(structBuilder.build()));
    } catch (InvalidProtocolBufferException e) {
      // Error reporting an error! FML
      System.err.println(e.getMessage());
    } catch (IOException e) {
      // Do nothing (there is not a JSON Payload)
    }

  }

}

This class add labels and transform a JSON String in a JSONPayload

You need to specify the LoggingEnhacer on the logback.xml file

<!DOCTYPE configuration>
<configuration>
  <appender name="CLOUD" class="com.google.cloud.logging.logback.LoggingAppender">

    <filter class="ch.qos.logback.classic.filter.ThresholdFilter">
      <level>INFO</level>
    </filter>
    <log>application.log</log> 
    <resourceType>gae_app</resourceType> 
    <!--  References to the LoggingEnhacer class -->
     <enhancer>[path_for_your_logging_enhancer_class]</enhancer>
    <flushLevel>WARN</flushLevel> 
  </appender>

  <root level="info">
    <appender-ref ref="CLOUD" />
  </root>
</configuration>

来源：https://stackoverflow.com/questions/50977940/how-to-use-stackdriver-structured-logging-in-app-engine-flex-java-environment

标签

java

google-app-engine