Wrap subprocess' stdout/stderr

Question

I\'d like to both capture and display the output of a process that I invoke through Python\'s subprocess.

I thought I could just pass my file-like object as named pa

Answer 1

Stdin, stdout and stderr of a process need to be real file descriptors. (That is actually not a restriction imposed by Python, but rather how pipes work on the OS level.) So you will need a different solution.

If you want to track both stdout an stderr in real time, you will need asynchronous I/O or threads.

• Asynchronous I/O: With the standard synchronous (=blocking) I/O, a read to one of the streams could block, disallowing access to the other one in real time. If you are on Unix, you can use non-blocking I/O as described in this answer. However, on Windows you will be out of luck with this approach. More on asynchronous I/O in Python and some alternatives are shown in this video.

• Threads: Another common way to deal with this problem is to create one thread for each file descriptor you want to read from in real time. The threads only handle the file descriptor they are assinged to, so blocking I/O won't harm.