Bash process substitution and syncing

Question

(Possibly related to Do some programs not accept process substitution for input files?)

In some Bash unit test scripts I\'m using the following trick to log and<

Answer 1

I sometimes put a guard:

: > >(sleep 1; echo a; touch guard) \
  && while true; do
    [ -f "guard" ] && { rm guard; break; }
     sleep 0.2
  done    

Answer 2

Insert a sleep 5 or whatnot in place of sync to answer your last question

Answer 3

In bash, a process substitution substitution command foo > >(bar) finishes as soon as foo finishes. (This is not discussed in the documentation.) You can check this with

: > >(sleep 1; echo a)

This command returns immediately, then prints a asynchronously one second later.

In your case, the tee command takes just one little bit of time to finish after command completes. Adding sync gave tee enough time to complete, but this doesn't remove the race condition, any more than adding a sleep would, it just makes the race more unlikely to manifest.

More generally, sync does not have any internally observable effect: it only makes a difference if you want to access device where your filesystems are stored under a different operating system instance. In clearer terms, if your system loses power, only data written before the last sync is guaranteed to be available after you reboot.

As for removing the race condition, here are a few of possible approaches:

• Explicitly synchronize all substituted processes.

  mkfifo sync.pipe
  command > >(tee -- "$stdoutF"; echo >sync.pipe)
         2> >(tee -- "$stderrF"; echo >sync.pipe)
  read line < sync.pipe; read line < sync.pipe
  

• Use a different temporary file name for each command instead of reusing $stdoutF and $stderrF, and enforce that the temporary file is always newly created.

• Give up on process substitution and use pipes instead.

  { { command | tee -- "$stdoutF" 1>&3; } 2>&1 \
              | tee -- "$stderrF" 1>&2; } 3>&1
  

  If you need the command's return status, bash puts it in ${PIPESTATUS[0]}.

  { { command | tee -- "$stdoutF" 1>&3; exit ${PIPESTATUS[0]}; } 2>&1 \
              | tee -- "$stderrF" 1>&2; } 3>&1
  if [ ${PIPESTATUS[0]} -ne 0 ]; then echo command failed; fi