Push startup-script logs to a separate file in gcp

问题

In GCP, For ubuntu - startup-script logs automatically pushed to /var/log/Syslog we might miss those logs due to log rotation if required after a long duration. Is there a way to redirect these logs to some another log-file? My startup-script is a simple bash script with multiple commands and can't redirect the output of individual command to a file.

回答1:

You can consider this solution:

• redirect outputs inside your startup-script to a dedicated startup-script.log file in /tmp directory
• install stackdriver logging agent
• add a specific configuration for your startup-script.log

Then you'll be able to browse your logs via GCP Stackdriver Logging console (or via gcloud command).

Screenshot of GCP Logging Console :

Stackdriver Logging will keep logs only for 30 days. For a long retention period, you can easily create a sink to export logs to a BigQuery table or a Cloud Storage bucket. Check official docs about exporting logs :

• https://cloud.google.com/logging/docs/basic-concepts#sinks
• https://cloud.google.com/logging/docs/export/configure_export_v2

Full code of a sample startup-script.sh:

#! /bin/bash

# install gcp logging agent
curl -sSO https://dl.google.com/cloudagents/install-logging-agent.sh
sudo bash install-logging-agent.sh

# setup a configuration for startup-script logs only
cat > /etc/google-fluentd/config.d/startup-script-log.conf <<- EOM
<source>
    @type tail
    # Format 'none' indicates the log is unstructured (text).
    format none
    # The path of the log file.
    path /tmp/startup-script-log.log
    # The path of the position file that records where in the log file
    # we have processed already. This is useful when the agent
    # restarts.
    pos_file /var/lib/google-fluentd/pos/startup-script-log.pos
    read_from_head true
    # The log tag for this log input.
    tag startup-script-log
</source>
EOM

# restart logging agent
sudo service google-fluentd restart

# redirect outputs to dedicated startup-script log
exec &>> /tmp/startup-script-log.log

# your startup-script content
# ...

echo "hello the world"

来源：https://stackoverflow.com/questions/58949139/push-startup-script-logs-to-a-separate-file-in-gcp