Repeatedly write to STDIN and read STDOUT of a Subprocess without closing it

Question

I am trying to employ a Subprocess in Python for keeping an external script open in a Server-like fashion. The external script first loads a model. Once this is done, it acc

Answer 1

Your external script probably buffers its output, so you only can read it in the father when the buffer in the child is flushed (which the child must do itself). One way to make it flush its buffers is probably closing the input because then it terminates in a proper fashion and flushes its buffers in the process.

If you have control over the external program (i. e. if you can patch it), insert a flushing after the output is produced.

Otherwise programs sometimes can be made to not buffer their output by attaching them to a pseudo-TTY (many programs, including the stdlib, assume that when their output is going to a TTY, no buffering is wished). But this is a bit tricky.

Answer 2

Thanks to this Answer, I found out that a slave handle must be used in order to properly communicate with the subprocess:

master, slave = pty.openpty()
tokenizer = subprocess.Popen(script, shell=True stdin=subprocess.PIPE, stdout=slave)
stdin_handle = process.stdin
stdout_handle = os.fdopen(master)

Now, I can communicate to the subprocess without closing it via

stdin_handle.write(input)
stdout_handle.readline() #gets the processed input