Saving stdout from subprocess.Popen to file, plus writing more stuff to the file

Question

I\'m writing a python script that uses subprocess.Popen to execute two programs (from compiled C code) which each produce stdout. The script gets that output and saves it to

Answer 1

As I understand it A program waits for B to do its thing and A exits only after B exits.

If B can start without A running then you could start the processes in the reverse order:

from os.path import join as pjoin
from subprocess import Popen

def run_async(cmd, logfile):
    print >>log, "calling", cmd
    p = Popen(cmd, stdout=logfile)
    print >>log, "started", cmd
    return p

def runTest(path, flags, name):
    log = open(name, "w", 1)  # line-buffered
    print >>log, 'calling both processes'
    pb = run_async([pjoin(path, "executable_b_name")] + flags.split(), log)
    pa = run_async([pjoin(path, "executable_a_name")] + flags.split(), log)
    print >>log, 'started both processes'
    pb.wait()
    print >>log, 'process B ended'
    pa.wait()
    print >>log, 'process A ended'
    log.close()

Note: calling log.flush() in the main processes has no effect on the file buffers in the child processes.

If child processes use block-buffering for stdout then you could try to force them to flush sooner using pexpect, pty, or stdbuf (it assumes that the processes use line-buffering if run interactively or they use C stdio library for I/O).