Prepend timestamp to each line received from stdin

问题

Hi there I'm trying to create logs using bash but the time I'm getting is wrong. I'm running this in background.

# log code here before executing curl script log "Yay I was started"

curl https://example.com/process | sed "s/^/$(date -u) /" >> /path/to/logs.log &

The time set after processing curl is the time it was executed not the time after I got the response. I need to get the time after the response.

Here's the sample output (Wrong response)

Fri Aug 21 01:43:12 UTC 2020 Yay I was started
Fri Aug 21 01:43:12 UTC 2020 Yay I was the response returned by the url

The url here https://example.com/process sleeps for 2 seconds but as you can see the time is the same for when it was executed and after it we got the response.

Correct Response

Fri Aug 21 01:43:12 UTC 2020 Yay I was started
Fri Aug 21 01:43:14 UTC 2020 Yay I was the response returned by the url

How can I solve this issue?

回答1:

Your issue is because sed "s/^/$(date -u) /" captures the date string at the time it is launched and places it at the start of each line. The date string is not updated for each incoming new line from the curl's response.

It is safer to use awk rather than sed, to prepend the date to each line, as it will capture a new date for each line.

Your original date format is locale formatted, local time. It has been kept here as-is, if it is your preference; but usually logs timestamps are better printed with the iso.org: ISO 8601 format, or an UTC relative Unix timestamp.

curl https://example.com/process |
  awk '{ printf("%s %s\n", strftime("%c", systime()), $0) }' >>/path/to/logs.log &

With an ISO-8601 timestamp:

curl https://example.com/process |
  awk '{ printf("%s %s\n", strftime("%Y-%m-%dT%H:%M:%S%z", systime()), $0) }' \
    >>/path/to/logs.log &

See man strftime for time formatting.

来源：https://stackoverflow.com/questions/63559785/prepend-timestamp-to-each-line-received-from-stdin

标签

bash

timestamp

stdin