How do I replace a substring by the output of a shell command with sed, awk or such?

Question

I'd like to use sed or any command line tool to replace parts of lines by the output of shell commands. For example:

• Replace linux epochs by human-readable timestamps, by calling date
• Replace hexa dumps of a specific protocol packets by their decoded counterparts, by calling an in-house decoder

sed seems best fitted because it allows to match patterns and reformat other things too, like moving bits of matches around, but is not mandatory.

Here is a simplified example:

echo "timestamp = 1234567890" | sed "s/timestamp = \(.*\)/timestamp = $(date -u --d @\1 "+%Y-%m-%d %T")/g"

Of course, the $(...) thing does not work. As far as I understand, that's for environment variables.

So what would the proper syntax be? Is sed recommended in this case ? I've spent several hours searching... Is sed even capable of this ? Are there other tools better suited?

Edit

I need...

• Pattern matching. The log is full of other things, so I need to be able to pinpoint the strings I want to replace based on context (text before and after, on the same line). This excludes column-position-based matching like awk '{$3...
• In-place replacement, so that the reste of the line, "Timestamp = " or whatever, remains unchanged. This exclused sed's 'e' command.

Answer 1

To run an external command in sed you need to use e. See an example:

$ echo "timestamp = 1234567890" | sed "s#timestamp = \(.*\)#date -u --d @\1 "\+\%Y"#e"
2009

With the full format:

$ sed "s#timestamp = \(.*\)#echo timestamp; date -u --d @\1 '\+\%Y-\%m-\%d \%T'#e" <<< "timestamp = 1234567890"
timestamp
2009-02-13 23:31:30

This catches the timestamp and converts it into +%Y format.

From man sed:

e

This command allows one to pipe input from a shell command into pattern space. If a substitution was made, the command that is found in pattern space is executed and pattern space is replaced with its output. A trailing newline is suppressed; results are undefined if the command to be executed contains a nul character. This is a GNU sed extension.

---

However, you see it is a bit "ugly". Depending on what you want to do, you'd better use a regular while loop to fetch the values and then use date normally. For example, if the file is like:

timestamp = 1234567890

Then you can say:

while IFS="=" read -r a b
do
  echo "$b"
done < file

this will make you have $b as the timestamp and then you can perform a date ....

Answer 2

As commented, use a language with built-in time functions. For example:

$ echo "timestamp = 1234567890" | gawk '{$3 = strftime("%F %T", $3)} 1'
timestamp = 2009-02-13 18:31:30

$ echo "timestamp = 1234567890" | perl -MTime::Piece -pe 's/(\d+)/ localtime($1)->strftime("%F %T") /e'
timestamp = 2009-02-13 18:31:30